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A STUDY ON PARAMETER IDENTIFIABILITY 

System identification is the process of modeling a system struc- 
ture, establishing a mathematical representation of that structure and . 
determining the values of the unknown coefficients, or parameters, 
from experimental input and output data records. In the broad sense, 
parameter identifiability may be considered as the mathematical assur- 
ance of the recoverability of the unknown parameters. Deterministic 
parameter identifiability pertains to systems in which no corruptive 
noise processes are present, while stochastic parameter identifiability 
treats those systems in which noise processes are present, either 
in the dynamics of the system itself, in the output observation 
process, or in both. 

A set of definitions for deterministic parameter identifiability 
is proposed based on the necessary injectivity of the mapping from the 
system composite input/initial condition/parameter space into the system 
output space. The equivalence of the proposed definitions and of 
various definitions previously developed is demonstrated. Determinis- 
tic parameter identifiability properties are presented based on four 
system characteristics: direct parameter recoverability, properties 

of the system transfer function, properties of output distinguishabil- 
ity, and uniqueness properties of a quadratic cost functional. 

Stochastic parameter identifiability is defined in terms of the 
existence of an estimation sequence for the unknown parameters which is 
consistent in probability. Stochastic parameter identifiability 
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properties are presented based on the following characteristics: 
convergence properties of the maximum likelihood estimate, properties 
of the joint probability density functions of the observations, and 
properties of the information matrix. 

Specific parameter identif lability properties for a number of 
specific systems and classes of systems are presented as theorems and 
examples. 
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1. INTRODUCTION 


The problem of system identification has recently become one of 
the more intensively studied and active fields of engineering and ap- 
plied mathematics research. The final objective of the system identi- 
fication problem is the production of an "accurate 11 model, or mathe- 
matical representation, to facilitate the study, understanding and, 
ultimately, the implementation, modification or improvement of a given 
system. The lack of such adequate system models is perhaps the great- 
est limiting obstacle preventing the application of powerful techniques 
of modem control, estimation, and filtering theory to such diverse 
areas as biological and human systems, ecological systems, socio- 
economic systems, and many other complex and multifaceted fields of 
endeavor not previously associated with exact, mathematical analysis. 

The system identification problem is generally considered to con- 

/ 

sist of three phases. In the first, or modeling phase, the basic mathe- 
matical structure of the system is determined. In determining this 
structure, varying degrees of a priori knowledge of the system may 
exist. As a minimum, the system observables (input and output variables) 
must be identified. Data records of these observables must be avail- 
able or must be obtainable through measurement experiments. If these 
data records constitute the total a priori knowledge of the system, 
the analyst is faced with the "total ignorance" or "black box" system 
Identification problem [2]. 

More conmon, however, is the "grey box" identification problem in 
which considerable knowledge of the system variables and internal 


2 


structural properties is known. In this case, the form of the describ- 
ing mathematical equations is known or may be readily deduced from the 
available defining, physical theory. Only this more common "grey box" 
identification problem will be considered in the following. The struc- 
ture or form of the mathematical equations defining the system under 
consideration will be considered to be known. 

Having determined the form of the defining system equations, it is 
then necessary to determine values for the unknown equation coefficients 
by an analysis of the available input and output data records.' The 
determination of these system parameters is the second phase of the 
system identification problem. Parameter estimation problems have 
been extensively investigated in the past, yielding well known results 
in such areas as least-squares analysis and curve fitting. Inherent 
to the parameter estimation problem, however, is the preliminary ques- 
tion of whether the system parameters can indeed be found under the 
given conditions and with the data available. That is, for the system 
as defined, are the system parameters mathematically identifiable? It 
is to this question of system parameter identif lability that this paper 
is addressed. 

The final step in the system identification problem is that of 
model verification. In this step, a final judgment is made of the 
model's ability to describe adequately the given system In terms of the 
objectives of the study. Such objectives might include the design of 
a control strategy for the system, the accurate simulation of the sys- 
tem, or the accurate prediction of the system response to varied inputs. 
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Obviously, the question of parameter identifisbility is criticsl 
to the generetion of s psrsneter estimstion algorithm. If the para- 
meters are not mathematically identifiabla, it is sanseless, at bast, 
to attempt the generation of such an algorithm. At worst, the imple- 
mentation of an estimation algorithm may generate spurious parameter 
values which may well lead to incorrect conclusions about the system 
properties . 

The question of parameter identifiability may also have a direct 
bearing on the other two phases of the parameter identification problem. 
An understanding of .the parameter identifiability properties of a given 
class of systems may guide the investigator in selecting an appropriate 
system model. Obviously, a model in which the system parameters are 
not identifiable must be rejected. If the form of the system is well 
defined by its physical properties, a knowledge of the parameter iden- 
tifiability properties of chat particular class of systems may lead to 
the proper choice of input signals or to the design of a proper output 
measurement scheme to insure system parameter identifiability. In like 
manner, the evaluation of the adequacy of the system model in terms of 
the ultimate investigation objectives must be considered in light of 
the limitations imposed by the parameter identifiability requirements. 
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2. BASIC CONCEPTS 

In lea broadest sense, parameter ldentlf lability may be considered 
as the mathematical assurance of the capability of determining unique 
values for the unknown parameters of a system from some set of input 
and output data records. That such assurance can not always be readily 
established is evident in the following simple examples. 

Example 2.1 . Consider the simple linear system described by the equa- 
tions 


x * ax + bu (1) 

y - cx (2) 

where the output y, the system state x, and the input u are each 
scalar-valued. The system parameters a, b, and c are to be determined 
from some set of input and output data. If the initial state of the 
system is assumed to be zero, the system input-output relation can be 
immediately written in the familiar form 

t 

y(t) « be J expla(t-T)]u(T)dT. (3) 

0 

The system parameters b and c appear only as the product be and i’Jius 
can not be separated and determined uniquely ftom the input-output 
measurements alone. Any parameter pair (b,c) which satisfies the 
relationship 

be - constant - k (4) 

will produce identical input-output records. If an attempt is made to 
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determine the values of b and c using a parameter estimation scheme of 
the smell variations type, such as the Newton or Gauss -Newton methods, 
the non-unique values generated by such a sequence, if indeed the para- 
meter estimation sequence converges at all, will usually be dependent 
upon the values assumed for the parameters at the first iteration. To 
obtain a uniquely described system, it is necessary to specify either 
b or c or to establish a defining relationship between the two para- 
meters. These properties are demonstrated in Figure 1. 

Example 2.2 . [3] Figure 2 shows a two-compartment model in which the 
concentrations of compartments 1 and 2 are designated and x^, res- 
pectively, and the input and rate coefficients arc designated as u and 
a^, a.y and a^, respectively. Such a model may be used to represent 
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Figure 1. Parameter relationships for Example 2.1. 
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Figure 2. A two-compartment model. 

i 

\ 

Che simplified oxygen transport characteristics of a puimonary/cardio- | 

vascular system in which the pulmonary subsystem is represented by 
compartment 1 and the cardiovascular subsystem is represented by com- 
partment 2. The oxygen levels or concentrations of the respective 
subsystems are denoted by and x^, and the oxygen transport coeffi- 
cients by a^, & 2 » a 3 > an< ^ a 4 * 

For a given oxygen input, u, the two-compartment model may be 
analyzed by monitoring the concentrations of either one or both of the 
compartments. If only the concentration of compartment 1 is monitored 
(i.e., x^ is the single output of the system), the equations describing 
the system may be written as 

x x « + a 2 )x 1 + a 3 x 2 + u (5) 

x 2 - a 2 x 2 - (a 3 + a 4 ) x 2 (6) 

y - • (7) 

It is desired to determine the rate coefficients, or system peri- 
meters a^, a 2 , a 3 and a^, from a set of input and output data. Assuming 

.1 

% 
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the system has achieved steady state, the input data ia directly related 
to the output data by the system transfer function which can be found to 

be 

s + a +a 

G(s) - -j - - (8) 

a + s(a 1 + a 2 + a 3 -»-a 4 ) 4 . (a^ a 2 )(a 3 + a 4 ) - * 2*3 

It ia obvious that G(s) will yield the same input-output relationship 
for any parameter combination for which 

a^ + a 4 ■ constant ■ c^ (9) 


a x + a 2 + a 3 + a 4 - constant * c 2 
and 


( 10 ) 


(a^ + a 2 ) (a 3 + a 4 ) - a 2 a 3 * constant » c 3 * ( 11 ) 

As Eqs. (9), (10), and (11) constitute an underde te mined set of three 
equations in four unknowns, the system parameters can not be uniquely 
determined from any given set of input and output data. To uniquely 
describe the system, either the value of one of the system parameters 
must be specified or a defining relationship among the four system para- 
meters must be specified. 

It is desirable that the system property of parameter identifl- 
ability be defined independently of the method used to estimate para- 
meter values. Parameter identif lability Is considered to be a property 
of the system itself. It is therexore necessary to explicitly define 
what is meant by a system or, more exactly, the mathematical represen- 
tation of a system and to enumerate the basic properties of such a 
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system [12]. In consonance with commonly accepted practice, the mathe- 
matical representation of the system is itself referred to as the 
system in the sequel. A general system is represented schematically by 
Figure 3 and mathematically by Eq. (12). 

w 

J_ 

— (Xg.i) 



Figure 3. A general system. 

1 * X(*0' -» £> + 1 (12) 

The quantities u, £, x, and d are the input, output, state, and 
parameters of the system, and w and v are noise processes. The system 
inputs and outputs and, indirectly, the noise processes are functions 
of time, t, normally defined on [0,*) for the continuous case and on 
{kTjk<I*J, I* being the non-negative integers, for the discrete case. 

To encompass these two possibilities, the domain of the system inputs 
and outputs is designated as T where TCR 1 * * 4 * and R + is the set of non- 
negative real numbers The system parameters, 2* * r * specifically time- 
independent ; that is, they are constant. 

The following sets and properties pertain to the system described 
by Eq. (12): 

J is the space of allowable input functions. An element u(*) of U 
is called a system input and, for any time tel, u(t) is called the 
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value of the input u at time t. The input, u, is of dimension r; speci- 
r it 

fically, u(t)eR , where R is the space of ordered r-tuples over the 
field of real numbers. 

/ is the space of output functions. An element of y is desig- 
nated as £(•) and is called a system output. For any given time tel, 
£(t) is called the value of the output £ at time t. The dimension of 
the output £ is m, and ^(t)€R m . 

X is the system state space for which an element xeX is known as 
a system state. X is n-dimensional and, in particular, XCR n . For any 
time teT, x(t)eX is called the system state at time t. In particular, 
x(0) is the initial state of the system and is designated x(0) = x^. 

The initial state of the system may be stated more generally as x(tg) 
for an arbitrary initial time t_. It should be noted that x-.eX and 
is therefore also n-dimensional, i.e., XqSR* 1 * 

ft is the space of allowable system parameters, and an element of 
ft is designated as _0 and is called a system parameter. At this point 
it should be noted that in generating properties of the general system 
or in manipulating system elements in generating parameter identifi- 
ability properties of the general system, arbitrary norms, designated 
by ||*|| , may be defined on the subject spaces as required, ft is a com- 
pact subset of R and hence, 0eR^ is p-dimensional. Arbitrary norms 
upon bP and R n may be considered to have been defined as required by 
the compactness property. Limited parameter identif lability results 
have been generated for more general parameter spaces. However, 
restricting ft to be a subset of R* 5 is not limiting for systems that 
model physically realizable processes, and parameter identif iability 
in generalized parameter spaces will not be considered. 
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w and v are generally additive, white, Gaussian noise processes 
affecting the system plant or state and the system output, respectively. 
The output noise process is specifically limited to being additive In 
nature while the plant noise process may be other than strictly additive 
in nature. Though not generally the case, either process may exhibit 
properties other than those of white, Gaussian noise. Each process 
may be correlated or statistically independent, the latter being far 
more common. 

A fundamental property of any given system in which the noise 
processes have been excluded is that, for any given initial time t^cT, 
for any given initial state Xq, for any given parameter 0eQ, and for 
any given input u (*)«U defined on some interval [tg,t], both the 
resulting state x(t) and resulting output £(t) at some later time t 
are uniquely determined. 

A minimal system or minimal realization is defined to be such 
that the dimension of the system state space X is less than or equal 
to that for any other equivalent system. Equivalent systems are de- 
fined to be those which generate identical outputs for any given input 
u«U. 

A rich diversity of systems and options are encompassed by Eq. (12) 
and Figure 3. The system itself may be linear or nonlinear with re- 
spect to its state, discrete- or continuous-time, single-input single- 
output (SISO) or multiple-input multiple-output (MIMD) . The form in 
which the parameters and state variables appear, the parameterization, 
may be canonical or noncanonical. The importance of a particular given 
parameterization is readily apparent by comparing the following Example 
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2.3 with Example 3.5 which involve two seemingly equivalent parameteri- 
zatlons of a linear, single-input single-output, second-order system. 


Example 2,3 . Consider the linear, SISO, second-order system character- 
ized by the transfer function 

a. 


H(s) 


(s - a 2 )(s - a 3 ) 
with the corresponding state space formulation 


(13) 


X 1 = a 3 x l + x 2 


x 2 = a 2 x 2 + a l u 


y - x. 


(14) 


(15) 


(16) 


The parameter to be determined is 6 * [a^, a^, a^J which is contained 

3 

in the parameter space * R. . 

The system may be written in an equivalent state space form as 


x = A(0)x + l>(0)u (17) 

y ■ £^x (18) 


where the parameterized, constant, system matrices are 


A(£) 


a 3 1 


1 

i ° 


■ « 

1 


, b(6) - 


> £ * 


0 a 2 

• 


a l 

m * 


0 


(19) 


To show that the given parameterization is not unique, and hence 

3 

not identifiable in ft = R , it is sufficient to show that there exists 
an invertible matrix P which creates an equivalently parameterized sys- 
tem through the similarity .transformation 
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-1 * 
PAP - A 

(20) 

. * 

P b - b 

(21) 

-1 * 
c p « c 

(22) 


it ic it 

where the matrices A , b , and c define the equivalent system. 
Consider the matrix 


P 


1 

a 3" a 2 


0 

1 


which yields its inverse 

m . 

1 0 

P _1 = 

V a 3 \ 

Then 


(23) 


(24) 


* 

A 


PAP 


, 

*-* 

o. 


” a 3 1 " 


' l o' 

a 3~ a 2 1 


0 a£ 


V a 3 1 

• « 


- « 


^ m 


it 

b * P b 



(25) 


( 26 ) 
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£ = £ P_ ^ * [ 1 0] 


1 0 


V a 3 1 


* [1 0] « c . 


(27) 


Note that the two parameterizations are identical except in the A 
* 

and A matrices which indicate that the parameters ^ an ^ a 3 are not 
uniquely identifiable. The parame terizations characterized by the two 
triples (a^, a a^) and (a^, a^, a^) are indistinguishable, their only 
difference being the basis coordination of their respective state 
spaces. 


In a system identification problem, the unknown quantities to be 
determined may include the system parameters 0_, the initial state x^, 
the covariance matrices of the plant noise process and output noise 
process, or any combination of these quantities. While it is often 
desired to determine x^ from the system input-output data, a sizable 
body of results has been produced for systems operating in steady state 
in which the initial states are therefore unimportant and may be assumed 
to have been zero, or for systems actually having a zero, or otherwise 
known, initial state. Such results lead to the concept of zero-state 
parameter identifiability. 

Consideration must also be given to the system input and its effect 
upon parameter identifiability. The input may be absent, present but 
unspecified, or present and designed to enhance the parameter identifi- 
ability properties of the system under study. Two commonly specified 
inputs are Gaussian white noise ana the unit impulse, the latter being 
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particularly attractive for linear, S1S0 systems as the impulse response 
defines the internal characteristics of such systems. While the char- 
acteristics of inputs ar usually specified to enhance the convergence 
properties of parameter estimation schemes as opposed to the actual 
parameter identifiability properties of the system, the presence or 
absence of some input is often critical to system parameter identifi- 
ability properties as seen in the following example. 

Example 2.4 . Consider the unity feedback system of Figure 4. 



Figure 4. A unity feedback system. 

The closed loop transfer function is readily found to be 

a a 

H(s) » -j i -j * . (28) 

s + a£S + (a^ + a^) s + 828 + a^ 

It will be explicitly demonstrated in Example 3.5 that the parameteri- 
zation of a system characterized by the given transfer function is 
identifiable and that the parameter values a^, a^, and a^ can be deter- 
mined. Since a^ ■ a^ - a^, the parameter values a^, 82 , and a^ are 
also identifiable. 

However, if the input is set identically to zero, u ^ 0, the 
following defining differential equation results 


* 



(29) 


y + a 2 y + («! + « 3 )y “ o. 

Obviously, any parameter pair (a^ , a^) such that 

a^ + a^ » constant ■ (30) 

yields an equivalent equation, and (a^, a^) is not uniquely identifi- 
able in the absence of a system input. 

While parameter identifiability is considered to be a system 
property and is therefore independent of the actual parameter values 
or the parameter estimation scheme used to recover these values, there 
exist derivative definitions based on the assumption of the existence 
of a scheme to generate a sequence of parameter estimates, which 
converges to the true parameter value. As these algorithms are 
usually of the small variations type, it is sufficient to consider 
parameter identifiability in terms of the uniqueness of the input/ 
output relationship generated by the true set of parameters as com- 
pared to all other parameters within some neighborhood of the true 
parameters. If this uniqueness property holds for all parameter space, 
the system is considered to possess global parameter identifiability. 
However, if this uniqueness property holds only within some limited 
neighborhood of the true parameter value, the system is said to possess 
local parameter identifiability within the defined neighborhood. Con- 
sequently, consideration must be given to determining this neighborhood, 
or region of parameter identifiability, in which a given parameter 
value is unique* Initiation of a parameter estimation sequence within 
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the region of parameter identifiability will ensure that the estimation 
sequence will converge to the true parameter values. 

Example 2.5 . The system of Example 2.3 was found to be not identifia- 
ble as the two parameterizations characterized by the ordered triples 
(a^, a 2> a^) and (a^, a^, a 2 ) were indistinguishable. If for a speci- 
fic input-output data record the following parameter values hold 

(®^i ®2* a 3^ * » ^2* ^j) (31) 

and 

(a^j a 2> a^) ® (k^» ^2^* (32) 

•fg 

then the partial (two-dimensional) parameter space, ft , of Figure 5 
can be drawn. 


a 3 
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•ff 

If Che partial parameter space is partitioned into and fl 2 aa 
shown, the system is locally identifiable in each of the two resulting 
•regions. Determining the respective regions of parameter identif lability, 
designated in Figure 5 as S[(k 2 , and S[(kj, k 2 );p 2 J for a parti- 

cular parameter estimation scheme, is a companion problem to the para- 
meter identif lability problem (see [13]). 

Consideration of the uniqueness properties of a given parameteri- 
zation within a neighborhood of the true parameter value leads to the 
concept of parameter distinguishability, the property within the given 
local neighborhood by which the true parameter values can be isolated 
or distlnquished from all other possible values in that neighborhood. 

This distinguishability property may be based upon different character- 
istics of the system under consideration, such as the ability of the 
true parameter value to generate a unique transfer function, a unique 
output, or a unique cost function for some parameter estimation scheme. 
This distinguishability property can then be related directly to the 
parameter identif iability property of the system. 

Perhaps most significant in the development of parameter identifi- 
ability properties of systems is the dichotomy of definitions and 
methodology required by the consideration of deterministic or noise 
free systems versus stochastic or noisy systems. For this reason, 
each is considered as a major category with all other subcategories, 
as discussed above, developed under each. 

It is generally desirable to develop parameter identif iability 
properties for an entire class of systems rather thai approach each 
system separately. However, due to the large number of classes and 
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options described by Figure 3 snd Eq. (12), a compact, comprehensive 
treatment of the parameter identifiability question has not been accom- 
plished to date* Instead, parameter identifiability definitions and 
developments have treated scattered, specific classes of systems, each 
seemingly with its own set of definitions and properties. It is the 
intent of this paper to consolidate these efforts into a unified and 
comprehensive overview. Therefore, basic definitions and approaches 
to the general spectrum of parameter identifiability problems are con- 
side red with the specific results for given classes of systems presented 

as examples of the basic philosophies. A chart is presented in Appendix 

« 

A which relates the characteristics of the covered system classes to 
the applicable theorems. 
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3. DETERMINISTIC PARAMETER IDENIIFIABILITY 
3.1 Concepti and Definition! 

One of the two basic categories of systems in the study of para- 
meter idencif lability is that in which the noise processes w and v of 
Eq. (2) are absent. The general deterministic system is then defined 
as shown in Figure 6 as 

X * K*o» i> (33) 

where the definitions of the variables, arguments, and corresponding 
spaces remain the same as given in Section 2. 


u 






Figure 6. A general deterministic system. 

For the given general deterministic system, the following general 
definitions are proposed: 

Definition 3.1 . A system is said to be Deterministically Identifiable 
if, for some u eu, the mapping £( *,* , is injective. 

Definition j.2 . A system is said to be Locally Deterministically Identi - 
fiable at (Xq, r^) if there exists an open sphere S(Xq, 2q,z) of 
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radius p > 0 centered at (x^, 0^) such that, for some ucU, the 
restriction of the mapping £(*,*, u):X*w-y to S(x^, £;p) 
is injective. 

Definition 3.3 . A system is said to be Zero State Deterministically 
Identifiable if, for some utU, the mapping £((),•, u):XXii-y is 
injective. 

Definition 3.4 . A system is said to be Locally Zero State Determinis- 
tically Identifiable at (0, 0^) if there exists an open sphere 
S(0, QqJP) of radiu i p ^ 0 centered at (0, 0^) such that, for 
some ue(j, the restriction of the mapping £(0,*, u):Xxij-y to 
S(0, S^jp) is injective. 

It should be recalled that _f, a function from X into Y, is said 
to be injective if for every x^, x 2 e X> then _f (x^) • £(x 2 ) l®plies that 
—l * —2* > l n terms of an arbitrary norm ||*|| defined on X 

and Y, _f is said to be injective if for every x^, X 2 eX > then ||l(x^) - 
,f(x 2 )|| * 0 implies that - x 2 || ■ 0 or x^ ■ x 2 . 

It will be shown in Che following that Definitions 3.1 through 
3.4 may be considered as the basic, encompassing definitions for deter- 
ministic parameter tdentif iability and that other definitions previous- 
ly propos<*d by other authors may be derived from, and hence are equiva- 
lent to, these basic definitions. 
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3.2 Deterministic Parameter Idcntlf lability from 
Direct Parameter Recoverability 


The injectivity of £_, us required by the basic definitions of 
deterministic parameter identif lability given in Section 3.1, directly 
implies the invertability of t and hence the direct recoverability of 
values for dn and x_. Using such an approach Staley and Yue [20] have 
developed tlic deterministic idcntifiability properties for a class of 
linear, constant-coefficient, stable, single-input single-output, dis- 
crete-time systems as described by the scalar difference equation 
n r 


x « ^.u ., J * 2, ••«, L (34) 

J i-1 J 1 i-1 1 J 

where the parameters and b^ arc unknown constants with |a^|<«, l*i*n, 
and )b |<», 1‘lSr. Although this system has a very particular struc • 
ture, it represents a rather large class of realistic problems. In 
particular, it may also represent the discretized version or linear, 
constant-coefficient, stable, SISO, continuous- time systems. 

The system of Eq. (34) may be stated in an equivalent input-state- 
output representation as 


z(j '+ 1) • A z(j) + b u , z(0) » Zq 


(35) 


x j * t T £(J) 


(36) 
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where 


^ 10 0 
« 2 0 1 0 


Vl 0 0 0 


0 

0 


(n X n); 


1 


» ■ 

b, 


1 

1 

b 2 


0 

• 

, (r X 1) ; h » 

• 

• 


• 

• 


• 

b 


0 

r 

u - 




, (n x 1). 


Staley and Yue (20) have proposed the following definition of 
identifiability for the given system which is readily seen to be a 
direct consequence of, and equivalent to, Definition 3.1. 


Definition 3.5 ( 201 . The system parameter 0 • (a^, .... a , b^, .... b^l 
and the tniciai state Zq are said to be Identifiable in the Deter - 
ministic Sense . IDS . if £ and Zq are uniquely determined from the 
observed input and output sequences ( u ^ ■ and | x j}> 0 £ j £ L - 1, 
for some finite integer L. 


The unknown parameter vector and the initial state may be 
expressed in terms of the input and output sequences using Toepiita 
matrices as 


\ * M + Mo 


(37) 
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where 


%, ■ t“ 0 i “i» — • Vj 1 . 0**1); 

* t*Q> • ••» i (1* ^ 2) I 


Sl 


i 

— nxn 


.-n,n 


, (L x n); 


and 


Sl - [S^, S^, ..., S^.S^, .... S^Jjh L x (n+ r), 


S L is Che LxL shift matrix defined such that S^(i,j) * 6^ j + ^. 

If Zq is known, such as when the system has an initial zero state, 
then Eq. (37) and Definition 3.5 together imply the following properties 
for Identifiability in the Deterministic Sense (IDS): 


Property 1 . Q is identifiable, IDS, if and only if the matrix is 

positive definite (or equivalently, has rank n + r) such that 
is uniquely determined. Note: Zq * 0. 


Property 2 . £ is identifiable, IDS, only if L 2 n + r. 

Property 3 . 0 is identifiable, IDS, for all L > Lq if it is identifi- 
able for L * Lg. 

By applying these three properties, the following theorems may be 
derived describing the identifiability properties of the class of 
systems described by Eqs. (34) or (35) and (36). 
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Th eorem 3.1 f 201 

In Che autonomous version of Che system described by Eq. (34) 
(i.e., b i s 0 or u i b 0, all i) , £ - (a^, ..., a^J is identifiable, 
IDS, if and only if (A.z^) is a completely controllable pair. Further, 
£ is identifiable, IDS, if and only if it is identifiable in 2n steps; 
i.e., L * 2n . 

In light of the universal Definition 3.3 for zero state determin- 
istic parameter identifi ability, it is proposed that the above theorem 
be modified to assert that _0 is simply "Zero State Deterministically 
Parameter Identifiable" as opposed to "identifiable, IDS". 

It should be noted, in particular, that the initial state is 
assumed known, that the parameter vector j) is limited to values of a^, 

1 s i s n, and that the input u^ a 0, all i or b^ a 0, all i. It may 
be recalled that the pair (A,Zq) is completely controllable if and pnly 
if the controllability matrix has rank n, 

rank( Z q, A jSq, ^ —0* •*** — ZqJ = n. (38) 

I 

Example 3.1 [20] . While it is easily shown that the initial state Zq 
X 

* [0, 0, ..., 1] always yields an identifiable system, consider the 
second-order system described by 


x. - (c + d)x. . + cdx. 0 = 0 
J J-l J-2 


( 39 ) 


with an initial state Zq * [1 - d] 1 . For this case. 


fc J r d l] 


A = 


L-cd 0J 
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and (A,£q) is not a controllable pair. The vector z^ la an eigenvector 
of A corresponding to the eigenvalue c. The state transition is con* 
fined to a proper subspace of R n which is invariant under A, and no 
information about d, the other eigenvalue of A, is contained in the 
output. 

For a non- autonomous, zero-state system the following theorem ap- 
plies. A similar modification of the terminology "identifiable, IDS" 
is proposed to achieve conformity with Definition 3.3. 


Theorem 3.2 f 20] 


If the system of Eq. (34) is stable and z^ * 0, then the parameter 

I 

vector £ » [a^, ..., a Q , ..., b^] is identifiable, IDS, from the 
input and output sequences | u j | > | x j| > 0 s j s I* - 1, if and only if 

(0) L S n + r, 

(1) are not all zero, 

n < 

(ii) the polynomials A(z) - 1 - ^2^ a^z- 1 and 

m j 

B(z) ■ j bjZ do not have a common factor, and 
(iii) Uj is not identically zero for 0 s j S L ■ n • r. 

When _Zq is unknown. Theorem 3.2 can be generalized by the following 
modifications: 


T 

S&- 


£ 

£ 


; hl. (2n + r) x (2n + r) 


fl T , a T T. 


The generalized result becomes: 
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Theorem 3.3 r 20] 

% 

If the system of Eq. (34) is stable, then the parameter vector 6 
and the initial state z^ are identifiable, IDS, from the input and out- 
put sequences |uj}, {x^J, 0 s j SL - 1, if and only if 

(0) L i 2n + r, 

(1) b i are not all zero, 

(ii) A(z) and B(z), as defined in Theorem 3.2, do not have a 
common factor, and 

(iii) the (2n + r) x (2n + r) matrix 



is positive definite where E^ remains as defined previously and 

2 nXy> 

h - [Sj^. ^ uj, Lx (n + r) . 

A change of terminology may also be made in Theorem 3.3 to conform 
with Definition 3.1. 

It may be noted that if s 0, all i. Theorem 3.1 applies. The 
proof of Theorem 3.1 is presented in Appendix B. Theorems 3.2 and 3.3 
are given without proof (see [20]). 

3.3 Deterministic Parameter Identifiability 
from the Transfer Function 

The injectivity of t required by the basic definitions of determin- 
istic parameter identifiability may also be interpreted as requiring a 
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unique relationship between the system input and output; that is, an 
injective mapping from U into y. Such a mapping is determined for two 
large classes of deterministic, linear, constant coefficient systems 
by their transfer functions. These two classes of systems may be de- 
fined for the continuous- time and discrete-time cases respectively by 


x(t,0) = A(61 x ( t,0) + B(0) u (t) (40) 

*(t,0) - C(0) x (t,0) + D(0) u (t) (41) 

and 

x(k +1,£) - A(9) x (k,0) + B(0) u (k) (42) 

X(k,0) - C(0) x (k,0) + D(0) u (k) (43) 


where x(*,JD crII > £(‘,.9)eR m , u(*)®0 and u(*)«R r , 0*flCR P , and A^), B (9) , 
C(0) , and D (0) are appropriately dimensioned constant matrices para- 
meterized by 0. For their respective transfer functions, H(s,0) and 
H(z,j3), the systems must be operating in the steady state mode in order 
to relate the system input u(«) to the system output £(*,.0). Hence, no 
information concerning the initial state Xq is available from the input- 
output data records. The systems may thus be considered to have a known 
initial state which may be taken without loss of generality as x^ » 0. 
The methodology employed and the results achieved are identical for 
both systems, except chat the Z Transform is employed for discrete- 
time systems and the Laplace Transform is employed for continuous-time 
systems. Therefore, only the continuous-time system of Eqs. (40) and 
(41) will be considered In the following. 
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The given system may now be investigated for properties of Zero 
State Oete minis tic Parameter Identif lability, as required by Defini- 
tion 3.3 and 3.4. For a given u eu, the system output may be written 
in the frequency domain as 

Y(s,0) * " A<£)3 ~ 1 B(D + D(0)| U(s,0) 

- H(s,9) U (s,0) (44) 

It is seen that H(s,0) must be (locally) injective (equivalently, 
unique) as a function of 8. Equivalently, from the definition of in- 
jectivity, for every 0^, 0, 2 *^» tiien * £( 8 >.®2^ ^l*® 8 that 

0^ * 0 , 2 * ^ at H( S »JL) 11,1186 b ® (locally) injective is evident from the 
uniqueness of the Laplace Transform/ Inverse Laplace Transform pair and 
from the fact that 

id'^s.e)} = f[0, 0, 5(t)] (45) 

where £ ^ is the Inverse Laplace Transform operator and 6(t) is the 
vector impulse function which generates the impulse response matrix. 

It should be noted that the vector impulse function 6(t) implies that 
an impulsive input is applied to each of the m input ports in sequence 
and that the resulting outputs each form a column of the impulse res- 
spouse matrix. Thus, these two particular classes of systems may be 
considered to require an impulsive input for the determination of para- 
meter identifiability properties and are so designated in the chart of 
system characteristics in Appendix A. 
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As implied in the sentence following Eq. (44), the simplest level 
of analysis may involve the direct inspection of g(s,£) to determine 
its uniqueness as a function of £. Such an analysis was conducted in 
Examples 2.2 and 2.3. 

Glover and. Willems [3], using the general concepts above, have 
produced for the two classes of systems under consideration a set of 
definitions and results. However, it will be readily seen that their 
definitions and results are a direct consequence of, and thus equiva- 
lent to, the basic definitions given in Section 3.1. Although the 
actual definitions and results of Glover and Willems [39] are repro- 
duced below for comparison, the obvious changes to produce conformity 
with the definitions of Section 3.1 are recommended. 

Definition 3.6a [91 . The linear, dynamic system characterized by Eqs. 
(40) and (41) is said to be locally identifiable from the transfer 
function at the point 9 q«Q if there exists a p > 0 such that 

(1) II ij. - iflll < P. II i 2 - iflll < .01 i 2 . <*«> 

and 

(2) £(0^ [Is - A^)]' 1 B(0 X ) + D(0 X ) - 

£(£ 2 ) CIS - A^)]' 1 B(6 2 ) + 3(0 2 ) (47) 

together imply £ x * * or seC ^ s ** > UA(£j)l} where 

\(>) denotes the eigenvalues of the respective matrix, C is the field 
of complex numbers, and ||*|| is an arbitrary norm. In consonance with 
the concept above. Definition 3.6a equivalently states that, in a 
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p -neighborhood of the true parameter 0^, there are no two systema with 
distinct parameters which have the same transfer function. 

As the system matrices (A, B, C, D)(0) are normally continuously 
differentiable with respect to £, the transfer function is meromorphic at 
Qq and Eq. (47) may be expanded in a power series to yield an equiva- 
lent definition. 

Definition 3.6b [91 . The linear, dynamical system characterized by 
Eqs. (40) and (41) is said to be locally identifiable from the 
transfer function at the point 0 q if there is an open sphere 
s (0».§oiP) c ft with radius p > 0 and centered at (0, 9, ) such that 


(1) 

^(o.e^p) 

(48) 

(2) 

D(0) “ 0(0^ 

(49) 

(3) 

* £(9o)- i( V- ( V , 

i - 1, 2, ... (50) 


together imply = Qq. 

Definition 3.7 . The Markov parameters for the system described by 
Eqs. (40) and (41) are defined in terms of the constant system 
matrices A (9) , B(0) , and C(6) av 

H “ £(0)A A (0)B(0), l « 0, 1, 2, ... (51) 

The Markov parameter matrix for the given system is defined as 
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G(0) - 


D T (8) 

[C(0)BC0)J T 

t£(i)A(i)B(i)] T 


[C(0)A 2n ' l (e)l(6)] T 


(52) 


It should be noted that Definition 3.6b is equivalent to the 
requirement that the mapping from the parameter space ft, or same subset 
thereof, into the Markov parameters also be injective. As a direct con 
sequence of the constant rank theorem [17], it can be shown that for 
an open sphere B(0,6^;p), centered at (0,9^) with radius p > 0, con- 
tained in ft and thus also an open subset of R*', then the mapping from 
s (°>£o;P) c U into the Markov parameters is locally injective if the 
rank of the Jacobian of the Markov parameter matrix equala p, the di- 
mensionality of the unknown parameters, 



As a direct consequence the following theorem is stated without proof 
Theorem 3.4 1 9 1 

The linear system characterized by Eqs. (40) and (41) in locally 
identifiable from the transfer function at 9 q<Q if the Jacobian of the 
Markov parameter matrix G(0) has constant rank p in an open sphere 
S(®>£o>P) of radius p > 0 centered at (0,0^); i.e., rank [3G(£)/d£] « p 


Employment of this theorem yields a relatively simple test for 
local parameter identifiability for the given classes of linear sys- 
tems. It ia constructive to consider two such examples. 
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Example 3.2 . Consider Che single-input a ingle -output, conClnuoua , 
constant, second-order system characterizing the two compartment model 
of Figure 2 and Example 2.2 


*1 ■ " (a l + 

a 2 )x 1 ♦ a 3 x 2 + u 

(54) 

*2 * a 2 X l “ 

(a 3 + a 4 )x 2 

(55) 

y « Xj. 


(56) 


It is required to determine if the four unknown rate constants a^, a^ , 
a^, and a^ are zero-state, deterministically identifiable. The four 
constant, system matrices defined in Eqs. (40) and (41) for the given 
systems are 



•< a l + a 2 } a 3 


i 

i 

A(9) « 


, b(0) - 



a 2 “< a 3 + V 


1 

o 

1 


c(6) - 



and D(0) » [0] 


(57) 


T 

where the unknown parameter vector 6 is defined as Q ■ [a^, a 2 , a 3 , a^] . 
The Markov parameter matrix for the second-order system (n * 2) is 
found to be 


£(D 


’ace> 


Xd) 

c(0)b(6) 


g 2 (0) 

£(£)A(i)b(£) 

- 

<l 3 CS> 

£(6)A 2 (i)b(e) 


£ 4 ( 9 ) 

£(i)A 3 (i)b(£) 


£5(6) 


(continued) 


(58) 
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0 

1 


(58) 


‘<*1 + V 
2 


a l + 2 V2 + a 2 + a 2 a 3 


-(a 2 + 3*1*2 + ^ a l a 2 + *2 + 2a l a 2 a 3 * 2a 2 a 3 
+ *2*3 * a 2*3 a 4 ) 


In order to check the local injectivity of the mapping of the 
parameter space into the Markov parameters and apply the results of 
Theorem 3.4, the Jacobian of the Markov parameter matrix is calcu- 
lated as 


3G(0) 

iF“’ 


3G^/3a^ 

dG^/da 2 

oG 2 /S a i 

322/3*2 

dG 3 /3a 1 

3G 3 /3a 2 

dG^/d*i 

354/3*2 

3^^ 

3Gj/3 * 2 


*i«*3 

35^3*4 

3£ 2 /S*3 

3S 2 /3* 4 

3G 3 /3* 3 

353/3*4 

aS4/3* 3 

354/3*4 

3G 5 /3a 3 

355 / 3*4 


0 

0 


-1 


2a^ + 2a2 

-(3a 2 + 6 * 2 + 3a 2 + 


(continued) (59) 
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0 

0 


-1 


2a^ + 2*2 + 

2 2 
-( 3a ^ + 6 * i *2 + 3 *2 + 2 * 1*3 


0 

0 

0 


+ 4a 2 a 3 


0 

0 

0 



I 

I 

I 


a 3 a 4> 


(59) 


*2 


• ( 28^2 + 2a 2 + 2a 2 a 3 + a 2 a 4 ) 
3G(9) 


* a 2 a 3 


3G(9) 

Clearly Che macrlx ia of rank 3, at most, while it must be 

of rank 4 for the mapping of the parameter space, ftCR , into the Harkov 
parameters to be locally injective. By the application of Theorem 3.4, 
the system parameters are not identifiable. This prediction of non- 
identif lability of the system rate coefficients was confirmed by a 
direct analysis of the system transfer function in Example 2.2. 


Example 3.3 . _ Modify the system of Figure 2 and Examples 2.2 and 3.2 
such that a^ » 0, yielding the following linear, constant, second-order 
system 

3^ - + a 2 )x 1 + a 3 x 2 * u (60) 

X 2 » a 2 x i" * 3*2 (61) 


y - 


(62) 
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(64) 

The Jacobian of Che Harkov parameter matrix can Chen be found Co be 

l 

SG(6) 1 

“SIT “ * l 1 

( 2a l + 2 *2 ) | 

"( 3a 1 + 6a l*2 + 3a 2 * U 2^) | 


(conCinued) (65) 
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0 

0 


-1 

(2a^ ♦ 2*2 + * 3 ) 

'( 3 *1 * 6 * 1*2 + 3 *2 + 2 * l a 3 + 4 * 2*3 + *3 


0 

0 

0 

*2 

( Z *l*2 + Z *2 + U 2‘l) 


which is clearly of rank 3 for all a„ + 0. Thua, the mapping of the 

4* 

3 

parameter space, QCR , into the Markov parameters is Injective for all 
*2 ^ 0, and, by Theorem 3.4, the modified system is locally identifiable 

3 

for all 9c R such that a 2 0. 

The prediction of the identifiability of the system parameters can 
be confirmed by direct analysis of the system transfer function which 
is found to be 


s + a 

H(s) ■ - ■ ■ -■■■ ■ * (66) 

a + sCa^ + a 2 + a^) + a^a^ 

It is readily evident that, for a given triple of constants (c^, C 2 » c^) 
an identical input/ output relationship will result for *ny combination 
of parameters in which 


( 67 ) 
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*1 * a 2 + a 3 " c 2 


( 68 ) 


*l a 3 


'3 ‘ 


(69) 


Since there are three equations in three unknowns, there exists a 

I 

unique solution for £ = [a^, a^, a^l ; in particular. 


a^ * c 2^ c i » c i * 0 


a 2 * ~ C 1 •' c 2 “ C 3 /C l ’ . C 1 * 0 


a 3«c 3 


(70) 

(71) 

(72) 


where the specific values of the triple (c^, c^, c^) correspond to the 
specific input/ output data record to be evaluated. Thus, as predicted 
by the application of Theorem 3.4, the parameters of the modified sys- 
tem are indeed identifiable. 

In Example 2.3, the parameter identifiability properties of a 
linear system of the type characterized by Eqs. (40) and (41) were 
investigated by demonstrating the existence of a similarity trans- 
formation which would transform a parameterized system into an equiva- 
lent system with different parameter values but with the same para- 
meterization. Glover and Willems [9] formalized the concept as follows. 

For the vector of true parameters 0^ and the space of invertible 
(nonsingular) n X n matrices PcGL(n) , the solution (P,0) for the 
following similarity transformation equations must be investigated: 

-1 


l ^ (i) P - A (Sq) 
P 1 (9) = B (Qq) 


(continued) 


(73) 
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C(0)P _1 - ccIq) 

D(0) ” DCSq) (73) 

It follows that the given system is (locally) identifiable from the 
transfer function at 0 q if there exist an open sphere S(0,6^;p)cn 
centered at (O,0 q) with radius p > 0 such that (I^^.8 q) is the unique 
solution of Eq. (73) in GL(n) x 3(0,0^). Sufficient conditions for 
such a unique solution to exist are given in the following theorem. 


Theorem 3.5 

Let the linear system characterized by Eqs. (40) and (41) be a 
minimal realization and define 


P A (0) P 




P B (3) 
C(6)P _1 


D(0) 


If there exists an open sphere S(O,0g,p) centered at (0,0^) with 


2 

radius p > 0 such that V,_ fl v£(P,0) has constant rank n + p at P = I 

- -nxn 

for all .068(0, 9^; p) , then the system is locally identifiable from the 
transfer function at 9 q« 

The matrix V^ p g^£(P,j3) evaluated at the point (I^^.3) is given 


by 
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' <*> ‘ i^Qlnxa | 

•JnxnQl ' (9) | 

•^ e) ©4xn I 


0 2 

-mrxn 


1 £' (6) 
II<®> 
I S c®) 
I I' (i) 


(74) 


2 2 

which is a (a + nm + m + rm) x (n + p) matrix and where^x) represents 
the Kronecker product [19] and X represents the standard Kronecker 
product matrix ordering. 

An application of Theorem 3.5 is given in the following example. 


Exaaple 3.4 . Consider the linear, SISO, second-order system 


x = A(0)x + b(0)u (75) 

y = £ T x (76) 

where 



I 

H 

O 

1 


1 

o 

1 


H* 

1 

A(6) . 


, b(9) =* 


> £ ■ 



-a 3 -a 2 

• m 


\ m 


1 

O 

1 


D = [0] and cl = £a lf a 2> a,jJ T . 
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Then 






0 

"®3 

0 

o' 


'o 

0 

1 

0 ‘ 

1 

” a 2 

0 

0 


0 

0 

0 

1 

0 

0 

0 

- a 3 


” a 3 

0 

" a 2 

0 

0 

0 

1 

" a 2 


0 

- a 3 

0 

“ a 2. 

• 





• 



a l 

0 

0 

-1 

0 


0 a, 
0 0 
■1 0 
0 -1 
0 0 


0 

0 

0 

0 

0 


0 

0 

0 


-1 

0 

a„ 

t 

1 

0 

0 

0 

0 

0 


0 

-1 

" a 2 

0 

0 

a l 

0 

0 

0 


0 

1 

0 

0 

0 

0 

0 

0 

0 


0 

0 

0 

1 

0 

0 

0 

0 

0 


0 0 0 
10 0 
0 0 0 
0 10 

0 0 0 
0 0 1 
0 0 0 
0 0 0 
0 0 0 


(77) 


which is of rank 2 +3*7 for all OtR . By Theorem 3.5, the system 
is identifiable from the transfer function (globally, since the rank of 
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<W&> 


is 7 for all 8eIT). 


It is constructive to consider a direct method of calculating a 
PeGL(n) satisfying Eq. (73). 


Example 3.5 . Consider the system of Example 3.4 and find a pcGL(n) 
satisfying the conditions of Eq. (73). Assume that such a matrix P 
exists, creating the equivalently parameterized system 


x* a A* (0) x* + b* (6)u 


* T* * 
y a £ X 


(78) 

(79) 


where 


and 



1 

f - 1 

o 

* „ 

'0 “ 

* 

V 


' p 

p 

ro 

i 

, b (0) - 

a 3 

L ^ 

£ “ 

0 

m m 


or 


P » 

P 11 

p 12 


. P 21 

p 22 

From Eq. (73) 


c P _1 

* 

a C_ 

-* c. ■ 

[1 

0] a 

11 < 


P 11 p 12 


P 21 p 22 


[ P ll p 12] 


(80) 


(81) 


(82) 


(83) 


which implies ■ 
Again from Eq. (73) 


1 and p, 


(84) 


or 


12 * 9< 


-I * * 

PAP *A - PA-A? 


i o ir o i 


P 21 P 22 


_a 3 ~ a 2 


" a 3 p 22 “ a 2 p 21 p 22 


"o 

1 * 


"l 0 " 


P 21 ? 22 

_ a l 

CL 

2_ 


P 21 P 22 

S 

_ a ! “ a 2 P 21 a 2 p 22 _ 


(85) 


whicn implies p 2 ^ ■ 0 and p 22 * Therefore, 
1 O' 


( 86 ) 


the identity matrix, thus establishing the uniqueness of the parameteri- 
zation of the system and confirming the results of Example 3.4. 

It should be noted that the state space formulation of Eqs. (75) 
and (76) yield the transfer function 

a. 


H(s) 


s + a 2 s + a^ 


(87) 


which appears essentially equivalent to that of Example 2.3. However, 
the system of Example 2.3 was found to be not identifiable, demonstrat- 
ing the importance of a given parameterization to the parameter identi- 
fiability properties of a particular system. 
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3.4. Deterministic Parameter Identifiability 
from Output Distinguiahability 

In Section 3.3, the injectivity of _f expressed in the bssic defi- * 
nit ions of deterministic parameter identifiability was interpreted as 
requiring an injective map from the system input function space U into 
the system output function space y as delineated by a system transfer 
function in terms of the parameters £. A very similar' method is to 
analyze directly the system output properties in terms of their injec- 
tivity properties with respect to the parameters £ as opposed to ana- 
lyzing the system transfer function which generates the system output 
for several classes of systems. By analyzing the output directly 
rather than limiting the analysis to the output-generating transfer 
function, a large set of system classes may be considered. 

The injectivity of _f requires, by definition, for any given u eU 
and 9 2 «U, that * jr(.,0 2 ) im P lies * i 2 * 11x1,8 be inter * 

preted as requiring that two outputs of the given system, corresponding 
to two different parameters 9^ and 0^ » J 4 @ 2 buC wl,t * 1 a common input 
u ell, must be different or distinguishable from each other for all 9^ 
and £2 ^ the parameter space for which the system is deterministi- 
cally parameter identifiable. 

Grewal et al. [10] , [llj and (12J have developed a set of defini- 
tions and results based on the distinguishability properties of the 
system output. However, as before and as indicated above, it will 
readily be seen that their definitions and results can be considered 
as a direct consequence of, and therefore equivalent to, the basic 
definitions of deterministic parameter identifiability in Section 3.1. 
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As in previous sections, Grevnl s definitions and results are recon- 
structed in the following for the purpose of comparison and, as previ- 
ously, the obvious changes are recommended to establish conformity 
with the definitions of Section 3.1. 

Grewal expanded the general deterministic system of Eq. (33) as 
follows: 


x(t»£) * £tx(t,0), u(t), t,0] ; x(t Q ) = Xg (88) 

l(t»9) * h[x(t,0), u(t), t,0] (89) 

where x(t,0)*R n ; £(t,_0)*R m ; u(t)eli; £eflcR p ; teR + ; ^:R n x u x Q x R + -• 
R n ; h: R n x U x £) x R + -* R m . The function £ is Lipschitz in x, continuous 
in u, and piecewise continuous in t; h is continuous in x and u and 
piecewise continuous in t. An equivalent formulation may be made for 
discrete- time systems. 

For parameter identifiability based on the distinguishability of 
the system output, the problem remains that of ascertaining whether or 
not the parameter values can be uniquely determined from a knowledge 
of the system input and output. In terms of a specific initial state, 
Xq, and a specific input, the output of the general system may 

be denoted as 

2(t,0) » H^, u(.),t,0]. (90) 

A single experiment may be defined in terms of a specified initial 
condition and input pair, £xq, u(*)J» and the resulting output H^, 
H(*)»t»£j» collection of all such allowable pairs is denoted by 
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t - ([xq, u(.)] :xQ«R n , u(0«tij (91) 

where U Is the spece of piecewise continuous functions. 

Having established the required background, the following defini- 
tions may be stated. 

Definition 3.8 [101 . The pair of parameter values (0^, 6 ^) , 0j«fl, 

is said to be indistinguishable if hJjCq, u(*)> t,£j 8 H|2q* «(•)» 
t,8j for all u(*)J«S and 0 s t s T. Otherwise, the pair 
(0^ & 2 ) is said to be distinguishable . 

Defining S^^jp) as an open neighborhood centered at (Xq,_ 9) with 
a radius p > 0, the definition of (local) parameter identif lability 
may be stated. 

Definition 3.9 [ 10] . A parameter set fl is identifiable at 0^ if the 
pair (8q , 6) is distinguishable for all 0e£), 0^0^, Further, a 
parameter set Q is said to be locally identifiable if there exists 
a p > 0 such that the pair (£q, 0) is distinguishable for all 

^(Xg.fljp), 0 # 0Q. 

It should be noted that the definition of identlfiabllity is inde- 
pendent of whatever method might be used to extract the unknown para- 
meter values from the input/output observation data. Further, although 
the class g of experiments is infinite, a finite number of experiments 
can be designed to distinguish between two systems, e.g., a zero initial 
state, impulse, or step response. 

The concept of parameter identifiabllity based on output distin- 
guiahability can be readily applied to linear, constant, dynamic systems 
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such as Chose represented by the, parameterized differential equations: 

i(t,6) - A(6)x(t,0) + B(6)u(t), x(t Q ) - 0 (92) 

Z(t,£) « C(e)x(t,6) + D(6)u(t) (93) 

where x(t,e)«R n , u(t)«R r , £(t,0)cR m , R 1 ** 11 , B:fl- R nxr , C:fl-.R mxn ‘ 

and Drfi-tR 0 **. The specifications for x and u cited for Eqs. (88) and 
•’89) continue to hold. It should be noted that the total number of 
unknown parameters, the dimensionality of the parameter space 0, equals 
the total number of elements in the matrices A, JJ, C and D; i.e., 
n(n + r+m)+mr»p. It should also be particularly noted chat the 
system is restricted to zero initial state analysis; i.e., x(tg) * 0* 
The solution to the system equations may be readily obtained by 
state transition matrix techniques. Output distinguishability of the 
given linear system may then be expressed in terms of this solution as 
given in the following theorem. 

Theorer. 3.6 f 101 

Tor the linear system described by Eqs. (92) and (93), the pair of 
parameter values (£^, £ 2 ) , £ 2 «fl, is indistinguishable ii and only 

if 

P* A(0.)(t-T) 

£(£].) J • !(£!>«< T)dr + D(£ 1 )u(t) 

/ A(6 )(t-T) 

■ £(£ 2 ) J • B(6 2 )u(T)dT + D(e 2 )u(t) 

0 

for all u(t)eU and for 0 £ t £ T. 


Output distinguishability of the given linear system may also be 
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expressed in terms of the system Merkov parameters as presented in the 
following corollary. 

Corollary 3.6 f 101 

For the linear system described by Eqs. (92) and (93), the pair 
of parameter vectors (0^, © 2 ) » l n< *i«tinguiahable If and 

only if 

£(6^ A 4 (6 X ) B (£ x ) ■ C(£ 2 ) A 4 (0 2 ) £ (0 2 ) , l m 0, 1, 2 

0(0^ • O(0 2 ) . 

For the given linear system, it is evident that indistinguish- 
ability implies that the Markov parameters of the system will be 
identical at different values of the system parameters, £. 

Parameter identifiability criteria for the given linear system 
may now be obtained by relating Theorem 3.6 and Corollary 3.6 to Defi- 
nition 3.10. 

Theorem 3.7 t 101 

For the linear system described by Eqs. (92) and (93), a parameter 
set Q is identifiable at 0^ if and only if - 

0eG 

D(6q) ■ D(£) , and 

* £<£>£'*' <£>£<£> . J - 0. 1, 2, ... 

together imply 0. » 6. 

™v " 
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It should be noted chat this identical result was presented as 
Definition 3.6b in Section 3.3, while the result here is obtained as 
a Theorem. It likewise follows Chat Theorem 3.7 requires that the 
mapping from the parameter space Cl into the Markov parameters be injec- 
tive, which gives rise to the following theorem which is equivalent to 
Theorem 3.4 of Section 3.3. 

Theorem 3.8 flOl 

For the linear system described by Eqs. (92) and (93), the para- 
meter set 0 is identifiable ‘at if and only if the Jacobian of the 
Markov parameter matrix G(0) has constant rank p in an open sphere 
SQtQ^jp) of radius p > 0 centered at (Xq,j 9); i.e., rank(dG(6)/d6] ■ p. 

Examples 3.2 and 3*3 pertain equally well to Theorem 3.8 as well 
as to Theorem 3.4 and will not be repeated. 

Distinguishabllity and identifiability results parallel to those 
of Theorems 3.6, 3.7, and 3.8 and Corollary 3.6 may also be obtained 
in the frequency domain for the linear system described by Eqs. (92) 
and (93) . These results are stated in terms of the system transfer 
function (see Ref. [10]). 

A commonly employed technique for the analysis of nonlinear sys- 
tems is the linearization of the system about an equilibrium or operat- 
ing point. As the identifiability of a parameter set Q has been de- 
fined at and in terms of a nominal parameter value 6^, the use of lin- 
earization techniques with this particular concept of parameter identi- 
fiability (teems particularly appropriate. Sufficient conditions have 
been derived under which the local identifiability of the parameters of 
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« linearized eye tarn will imply local ldentlf lability of the original 
nonlinear ayacem. 

Conalder the general ayacem deacribed by Eqs. (88) and (89) with 
the aaaunpclona and raacriceiona aa given. Lee 


x(t,0) - x^c.0) + floc(t,0) 

(94) 

*(t»£) * ^(t.®) + fc(t»£) 

(95) 


and 

i * ®q + W (96) 

where ^(t,^), ^(t.B), and 0^ are equilibrium or operating point 
values of che system state, system output, and system parameters, re* 
apectively, and 8x(t,0), 6jr(c,0), and 86 are perturbations onx^tt,©), 

and 0 q, respectively, Equations (88) and (89) may now be 
rewritten as 


*( c »®) " i (C) ‘ c * + 60]» £(t 0 ) - *q (97) 


With x ■ 


X(t»£) - li^t,^ 80) u(t), t, 0Q + WJ . 

u(t), t»£o]> Eqa. (97) and (98) become 
6 *( 6 ^, 80 ) + 

J^.uCt) ,t,0Q 86 


(98) 


«£<t, V «> * If 


66 




^(fyio'AS) • 0 (w> 
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dh 


feet, e 0 , M) - 


66 + . 


fe(t.io.fe) + dh 

V- (t) »**£<) ae 

+ i 2 ( fe'fe»t) (100) 

«*“fi l (fe.66,t) «ndi 2 (&c,6e,t) r.pr...nt higher ord.r u™. 

The linearized syscem equation* then can be written aa 


feo (t ’£o»fe> " |f 


For brevity, denote 


feo(t,6Q,66) ^ a* 

V^tj.t.SQ as 

. |= J * s o (t -'2o •»£> + & 

V- (t >» c *.§o ae 


fe* feo (t O’^0»fe> * 0 
V- (t) ' c *2o (10l) 


6£ 

2Se.»liCc) . £,©0 


. ( 102 ) 


• it i 


ie^et),^ 


c(t,e o) - | 




(103) 


(104) 


- ft 




(105) 


Then the solution to the linearized system equations may be written 
directly as 



( 106 ) 

where se^.T,^) is the system state transition matrix. It should be 
noted that 66 represents a vector of parameters. 
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t 

Denoting CCt,^) J* cg(t,T ,6 ^)B(t , 0g)dT + D^.Sq) as K^t,^), 

an m x p time-varying matrix, Eq. (106) can be rewritten an 

(107) 

It can be seen from Eq. (107) that the parameters 60 are identifiable, 
that is, are uniquely recoverable from Eq. (107), only if the mapping 
of the parameter space 60«fl into the system outputs 9 ^, 60 ^) is 

injective for a given input and initial condition. Such will be the 
case if and only if the columns of N(t,6^) are linearly independent. 

It can be shown [4, p. 75] that the columns of N(t,0g) are linearly 
independent if and only if the Granmian is nonsingular; i.e«, 

J* C H T (T,0 o )»(T,e o )dT > 0 . (108) 

c o 

The above results may be summarized in the following theorem. 
Theorem 3.9 [ 10] 

Consider a nonlinear system with a state differential 
equation 

£(t,0) * £[x(t,0),u(t),t,0] , x(t Q ) - Xq (109) 

and a linear output equation 

Z(t«S) * i»lx(t*£) ,u(t),t,0] (110) 

where the functions £ and h possess continuous partial derivatives 
with respect to the components of x and 0_. The linearized state dif- 
ferential equation about (x g , 0^) is 


iMaarismiatfi 




54 


6*o<t,eo ,66) - Act^^ct.eQ.aej+Kt^oe, « o 

(in) 


and Che ouCput equation is 


•AS) * £< t »®o ) ^o <t *^o ,6 - ) + (U2) 


where Che matrices A(t,_0Q) and B(t,3^) are Che Jacobians of £, and 
£(^>0^) and D(t,6^) are the Jacobians of h with respect to x and £, 
respectively, each evaluated at ^x^uCt) »C,0 qJ. Let 

t 

M(t,0o) - C ( t,0Q) J* sB( t .T»£ 0 ) a (T,V dT + £ (t ’V < 113) 

*0 

where g>(t,T,0g) is the transition matrix of the linearized system 
of Eqs. (Ill) and (112). 

Then, for the given input u(t)e{J, if 


S r (T«e o )fl(T,0 o )* T > 0 


(114) 


the parameters Qeft of the nonlinear system can be locally identified. 


Another sufficient condition has been derived for the parameter 
identifiability of a nonlinear system for any input 11 sufficiently 
close" to a specified input. Assume that the parameters, deft, are 
fixed and consider only the effects of small perturbations in x and/or 
u on the system motion. Then, a sufficient condition for the identifi- 
ability of the parameters of a given nonlinear system is given in the 
following theorem. 

Theorem 3.10 [10] 

Consider a nonlinear system with a state differential equation 
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x(t,9) » ^[x(t,0),u(t),t,0] , x(t Q ) - Xq (115) 

and a linear output equation 


Z( t,6) - h[x(t,0),u(t),t,0] 


(116) 


\A lere the functions £ and h possess continuous partial derivatives 
with respect to the components of x and u at and u^, respectively. 
The linearized state differential equation about (x^,Uq) is 


fi*Q(t»0) ■ A(t,9)&c (J (t,0) +B(t J 9)fiu(t), fi*^) » 0 

(117) 


and the output equation is 


6Zo(t,9) = C(t,0)&c o (t,0) + D(t,0)6u(t) (118) 

where the matrices A(t,0) and B(C,0) are the Jacobiana of £, and 
C(t,0) and D(t,0) are the Jacobians of h with respect to x and u, 
respectively, each evaluated at (x^.UQ.t.0) . 

Then, if the parameter set ft of the linearized system of Eqs. 
(117) and (118) is identifiable at 0,, the parameter set ft of the 
nonlinear system of Eqs. (115) and (116) is also identifiable at £. 

Example 3.5 flOl . Consider a single input/ single output, nonlinear, 
second order system 


X x (t) » -a 2 x 2 (t) 

• , v 3 2 

(119) 

X 2 (t) * " a i x l (t) " a 2 X 2 (t) + u (t) 

(120) 

y(t) » x x (t). 

(121) 
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2 

Note the parameterization 6cQ « [<a^, a^)eR : 4 0, / 0] . For a 

constant input u^ 4 0, the equilibrium states of the system are found 
to be 


1 

ID* 

*-■ 

1 


'(■JO 1/3 ‘ 


Z= 


1 — 

X 

ID 

fo 

1 


v — 

O 


Nominal values of the parameters a^ and a^ are denoted by a ^ QOm a nd 

a 2nom* ** et ^*1’ ^2* ^ P ert “ r ^ at i° n8 on x e ^» x e 2> an< ^ u q» 

respectively. The linearized equations are then found to be 


ei(t,6) 

6*2 ( c «£) 




6x 2 (t,9.) 



(123) 


6y(t,0) = [1 0J 


(t,0) 

5x 2 (t,e) 


(124) 


For this linearized set of equations the Markov parameter matrix is 
calculated as 


G(£) 


" 2u 0 a 2 

„ 2 
2u 0 a 2 


■ l2a lVel 


-2u„a 


3 
0"*2 


( 125 ) 


and its Jacobian as 
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SG(0) 

ae 


o 

o 

o 

o 

,,2 2 
-12a~x . 

2 el 


0 

0 


■2u, 


4 * 2"0 

- 6 “o*2 


( 126 ) 


2 

The rank of the Jacobian la clearly equal to 2 for all QtR , and, by 

Theorem 3.8, the parameterization of the linearized system la identi- 
2 

fiable for all 6e& . By Theorem 3.10, the parameterization of the 

2 

original nonlinear system also is identifiable at any 6 » (a^, a 2 )«R • 


3.5. Least Square Deterministic Parameter Identifiabilitv 

The definitions of parameter identifiability stated above have 
been independent of the method used to recover the unknown parameter 
values. Bellman and Astr’dm [3] and KRrtensson [15] have proposed an 
algorithm-oriented definition called locally least square identifi - 
abilitv in which experimental data are combined with £ priori or as- 
sumed knowledge of the system structures. 

In many estimation methods of the small variational type, the 
values of the unknown parameters are chosen to minimize a quadratic 
cost functional of the form 

T 

J t ( 8) - <f II 2( T >!) - X n ( T )|| 2 dT, T > 0 (127) 

*0 
or 

K 

“ k f 0 II * (k >£> “ V^ll 2 * K - °> 1 (128) 

where y^( O is the measured system output for some given input u and 
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||.|| denotes the Euclidean norm imposed upon R m . Local least square 
identif lability is then defined as follows. 

Definition 3.10 f3l . The system parameters 0 are said to be locally 

least square identifiable if and only if J(0) has an isolated local 
minimum st £ « ^ and J(0 q) “ 0. If the minimum is global, the 
parameters are said to be globally least square identifiable . 

It should be noted that, contrary to later authors. Bellman and 
Astrom [3] did not require J(£q) = 0. While this may be a valid omis- 
sion for the extension of the concept of least square identifiability 
to the stochastic case, for deterministic systems the requirement that 
J(6 q) * 0 must hold necessarily as 




(129) 


In terms of Eq. (33), Eqs. (127) and (128) become 


J T (£) - S T lli( T »5o»i»ii) - ifa, T > 0 


(130) 


J K (6) =J 0 || f (k.XQ.e.u) -i(k,0)|| 2 , K= 0, 1, .... (131) 

where the specific time dependencies have been added to the expressions. 
As equivalent results may be obtained from either formulation above, 
consider only Eq. (130). Since the integrand of Eq. (130) is non- 
negative, the requirement that J(@q) = 0 implies that 


I i(T,2o.e Q .u) “ *< T >£>II * 0 


(132) 
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when 


II V HI " 0 < 133 > 

which is an equivalent statement of the definition of injectivity of 
_f with respect to Q. Clearly, then, the definition of least square 
identif lability is derivative upon, and equivalent to, the definitions 
of deterministic parameter identlfiability given in Section 3.1. 

A sufficient condition to insure least square deterministic para- 
meter identifiability is found in a restatement of the implicit func- 
tion theorem. 

Theorem 3.11 f 3] 

A sufficient condition for the parameter 0 q to be locally least 
square identifiable is that there exists an open sphere S(6 q,P)c0cr^ 
with radius p > 0 centered at 0^ such that J^(Q^) > 0 and that the 
(p x p) matrix of second-order partial derivatives with respect to the 
parameters, J^(0), is positive definite for all 0cS(0Q,p). 

Grewal and Glover [12] have established the equivalency of least 
square identifiability and of identifiability based on output distin- 
guishability. Such equivalency will not be demonstrated here since 
mutual equivalency of all deterministic identifiability definitions 
has been established through the definitions of Section 3.1. 

3.6. Comments on Deterministic Parameter Identifiability 


It should be noted that although a number of seemingly different 
definitions for deterministic parameter identifiability have been 
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presented by different authors, it has been shown that they may be 
derived from, and hence are equivalent to, the definitions presented 
in Section 3.1 which require the injectivity of the function f. (Xq»,£q»u) 
of Eq. (33) as the criteria for deterministic parameter identifiability. 
It is thus recommended that the four definitions of Section 3.1 be con- 
sidered as the general definitions for deterministic parameter identi- 
fiability and that the resulting theorems and system properties derived 
in Section 3 be directly related to and derived from them. 

It may be further noted that although certain system-specific 
properties and theorems have been generated, such as Theorem 3.2, there 
exist only three general methods of establishing deterministic para- 
meter identifiability. These three methods essentially require the 
certification 

(1) that the parameters are uniquely recoverable from the system 
mathematics (e.g., recoverable by Cramer's Rule, uniquely 
recoverable by observation of the transfer function, etc.); 

(2) that the Jacobian of the Markov parameter matrix is of con- 
stant rank equal to the dimensionality of the parameter space 
in some open neighborhood of the true parameter value; or 

(3) that the second partial derivative of a quadratic cost func- 
tional with respect to the parameters is positive definite 
in some open neighborhood of the true parameter value. 

The first two methods are particularly suited for analysis of 
linear systems while the third may be employed with either linear or 
non-linear systems. 
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4. STOCHASTIC PARAMETER IDEM! IF LABILITY 
4.1. Concept* and Definitions 

The second of the. two basic categories in the study of parameter 
identifiabllity has been developed for the class of stochastic systems 
in which one or both of the noise process, w and v, of Figure 3 and 
Eq. (12) are present. The general system as it was given in Section 2 
is to be considered in the following. 

In Section 2, parameter identifiabllity was considered in the 
broadest sense as the mathematical assurance of the capability of 
determining unique values for the unknown parameters of a system from 
some set of input and output data records. In Section 3, this defini- 
tion was restated for the deterministic (or noiseless) category of 
systems in terms of the injectivity of the mapping, for some accept- 
able input, of the system composite parameter and initial state space 
into the system output space. The injectivity of this mapping insured, 
among other properties, the existence of the functional inverse and 
hence the recoverability of the parameter values. Stochastic para- 
meter identifiabllity may be considered as the stochastic analog to 
deterministic parameter identifiabllity. That is, stochastic para- 
meter identifiabllity is the mathematical assurance of recovering 
from noisy observation data the system parameter and/or initial state 
values of interest in soma probabilistic sense. This may be further 
interpreted as assuring the existence of a sequence of estimates of 
the unknown quantities which converges in some probabilistic sense 
to the true values of the quantities. 




* 
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In Che presence of noise, Che ouCpuC dace of Che given syscem be- 
comes s sequence of random variables 

Z 2 « **** k * 1, 2, ... or . (134) 

Based on Chis observaCion sequence, in conjuncCion with an actual or 
assumed knowledge of Che scruccural properties of the system, the para- 
meter identification problem becomes that of generating a sequence of 
estimates of the unknown system parameters which will converge in a 
stochastic sense to the true parameter value. Such an estimation 
sequence is a measurable function of the observation or output data 
and is denoted by 

A A 

•••» > k * i, 2, ... or [9, k«l * (135) 

The true parameter value is denoted by 0^. The vectors 0 ^ and 9^ 
belong to 0, the space of allowable parameters, which may be considered 
to be a subset of R**, the space of ordered p-tuples. Although results 
have been obtained for more general spaces, the restriction of 0 to R** 
is not generally limiting for realizable, physical systems. 

Definition 4.1. A sequence of random variables Is to con - 

verge to z in probability (converge stochastically to z ) if for 
every e > 0 

lim Pr Q - zj * « 0 

where Pr is a probability measure defined on R r . 
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Definition 4.2 . An estlmetion sequence [8^] of which converges 
stochastically to 3^ is said to be consistent In probebllitv : i.e., 

CD 

[9^] is a consisten t estim ate for 6^ . This property may be 

A 

denoted by ^ <8 0^. 

Stochastic parameter identif iability of the initial state end 
system parameter 9^ can now be defined in terms of the consistency of 
the sequences of their estimates. 

Definition 4.3 . The initial state, x^, end the system parameters, £, 

are said to be stochastically Identifiable if there exist sequences 

CD A CD 

of estimates [x^j and which are consistent in proba- 

bility; i.e., x^Xq and 0^^. 

It may be noted that, as with the definition of deterministic 
parameter identif iability, the definition of stochastic parsmeter 
identiflablllty is independent of the method chosen to generate the 
estimation sequences. However, the standard method normally chosen 
for generating these sequences has been the maximum likelihood esti- 
mate method. For simplicity, consider the case when only 8^ is unknown. 

CD 

be a sequence of random variables with given joint 

probability density functions p^^, •••» £) > k » 1, 2 

which are of known functional form but which depend upon the unknown 
parameter vector dcQ, the allowable parameter space. Thus, there exists 
a family of joint probability density functions denoted by [p^C^, .... 

9):£efl, k ■ 1, 2, ...]. For each value of 0cQ, there corresponds 
one member of the family, specifically Ip^^* •••» 2^; £):k * 1* 2 , ...J. 
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which la a sequence of Joint probability denaity functiona indexed by 
k - 1, 2, . ... and parameterized by 6. The member of the fwiiy corre- 
aponding to the true parameter vector la denoted by [p k (x^, . .., 

6g):k ■ 1, 2, The aequence of maximum likelihood estimates, then, 

ia obtained by aelecting 6^ auch that 




* • $ 






max P k C^i. 
6cQ K 1 


V I); 


1 , 2 , 


( 136 ) 

The eatimatea may be expre8aed more explicitly in terms of the maximum 
likelihood equation 


S log p k (x lt .... 0) 

06 


k • 1, 2, . 


( 137 ) 


Under certain xeatrictions on the joint probability densities of 
the observations, Wald [ 24 , 25 ] has shown that the maximum likelihood 
equation has at least one root which ia a consistent estimate of the 
parameter 9 to be estimated. That is, if a given system satisfies the 
restrictions such that a consistent sequence of estimates for an un- 
known parameter exists, the method of maximum likelihood estimation 
will surely produce that aequence. 

OD 

Let [2^]^.} he a aequence of independent, identically distributed 
random variables with joint probability density function p(Xp ..., x^i 
_6) , k ■ 1, 2, ..., parameterized by the unknown paraneter _0eGCR p , where 
Q ia the allowable parameter space. The probability density function 
is denoted by p(£; < 6) and the corresponding cumulative distribution 
function is denoted by F(x;0); i.e., F(x;0) • Prlx^ * x) • An arbi- 
trary norm on R p ia denoted by ||*||. The following notation ia used. 
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P (X>!>P) “ P(x;®'). i«G and p > 0 

II® - I'll < p 


(138) 


*(£»*) ■ „ «*P P<2li). r > 0 

II ill > r 

* |p(i;i»p) , p(*;i,P) > i 

p (x;i»p> • j 

\ 1 , otherwise 


(139) 


(140) 


* ♦ (*.*) , *Qr,r) > 1 

♦ (l.r) - 

1 , oth«rwi*« 


(141) 


Weld 1 a restriction* may now be expreaaed aa the following eight 
assumptions. 

Aaaumption 1 . F (£-,$) la either discrete or is absolutely continuous 
for all £cft« 


Assumption 2 . For sufficiently small p and for sufficiently large r ( 

J* log p*(x;6,p) dF (Xiio) <® 

-00 ^ 

and 

00 

X log i|f*(£;r) dF (x;$) <® for all 0efl. 

-00 

Aaaumption _3‘ 1 £ jll® ifc * i» than jlim ptjr;^) • p(^;0) for all £ ex- 

cept perhaps on a set whose probability measure is zero according to 
the probability distribution corresponding to 6«. 


Assumption 4 . If then F^;^) - F&iig) for 4t l* 4 * 6 one 

value of £. 


f 


Assumption 5 . If lia H&. || ■*, then lim p(£;6. ) » 0 for every y except 
k -* ® * k - ® ■** 

perhaps on a fixed set whose probability Measure is zero according to 
the true p-?r«aeter 6^. 

00 

6, £ I log PdjSoJldFdiV <• . 


7. Q is a closed subset of R p . 


Assumption 8 . p(£i£»P) is a measurable function of £ for _6eft and p>0. 

Succeeding work in stochastic parameter identifiability has been 
primarily accomplished in applying or re- interpreting in more useful, 
system-oriented terms Wald's restrictions on the Joint probability 
densities of the system observations. A very simplified application 
of the maximum likelihood estimation method is given in the next 
example. 


Example 4.1. Consider a single observation y of a parameter a corrup- 

2 

ted by additive, zero-mean, Gaussian noise with variance a : 

n 


y ■ a + n. 


(142) 


The probability density function of the noise is 


1 -n 

1/2 ex P 2 

(2TTr' <j 2c 

v n n 


(143) 


Since y > n - a, the probability density function of y conditioned 
upon CL is 


(2tt) a/ X 


and 
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log p(y;a) * Constant^ • 
The maximum likelihood equation ia 


. Conatal , t2 . 


(V - a) 


2a: 


ll. -&I 


( 145 ) 


( 146 ) 


and the best (maximum likelihood) estimate d for the parameter a is the 
observed value y; i.e., 


a * y. 


( 147 ) 


4.2. Stochastic Parameter Identifiabilitv from the 
Properties of the Maximum Likelihood Estimate 

Stochastic parameter identifiability results may be derived di- 
rectly from the convergence properties of the maximum likelihood esti- 
mate. Though methods and results developed at later times are easier 
to apply to a given system, Aoki and Yue [1], using this direct analy- 
sis approach, established results for a class of linear, stable, con- 
stant-coefficient, discrete-time dynamic systems with both plant and 
observation noise present. Because of the importance of these results 
and as an aid to understanding the properties of the maximum likelihood 
estimate, the major parts of their results are reproduced in the following. 

A class of systems nearly identical to that of Section 3.2, except 
with the addition of noise sources, was investigated. As with the pre- 
vious formulation, this class of systems represents a large class of 
realistic problems. Tha system class is characterized by the equations 
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where 


*(j + 1) • A z.(j) + b u., 2 ,( 0 ) » 2 ^ 


x j * *(J) 


(148) 

(149) 


1 0 0 ... 0 

0 1 0 ... 0 


-a , 0 0 0 ... 1 

n-1 


-a 


0 0 0 ... 0 


(n X n) ; 


, (n x 1); h 


1 

0 


(n x 1); 


and Che parameters a^ and are unknown constants with |a^| <« and 

jb^| <ob , 1 s i s n, 

Output observations, y., are made with additive noise 


y . » x. + TJ. . 

J J 3 


(150) 


The output noise process } is restricted to be independent and iden- 

2 

tically distributed as zero-mean, normal with variance a . 

As in Section 3.2, the system may be alternately represented by 


+ ill V)-1 ■ 1*1 


y j “ iSl (x i + V » j ■ 0, i, 2, .... L-l 


(151) 

(152) 
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or with Toeplitz matrices as 


A.X. » B t u t + E.Z.. 
— L— L —L-L — L-0 


or 


where 


*L * Sl + h 


— L = -L- + -IrO 


■^L * — L + h 


x * [x Q> x x> .... x L-1 ] , (L x 1) ; 


1 T * [y Q , y x » y T _,] , (L x n ; 


L-l 


-L * £ u 0’ U l’ U L-1^ ’ (L X 1) 

. T 


(L x 1) ; 
T 


= ^1’ ***’ ^L-l^ 

_9 ® [a^ , a .^ » • • • > , b^, i>2 » • • • > "o 

— L * — L + iil a i“L * (LXL) > 
i=l b.s[ , (LXL); 

, (L X n) ; 


*L 




■^nxn 


L-n,n 


(153) 

(154) 


(155) 

(156) 


(2n x 1); 


■“L * L* — l— l’ * * * * — jj—’L* “L^L* * * * * ”L”L^ * (1* ^ 2n) ; 


and S T is Che L x L shift matrix defined such that S T (i,j) * 6 

~ L ~L i>j+l 
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For the deterministic portion of the system [Eqs. (148) and (149) 
or Eqs. (151), (153) or (155)] results may be stated which are identi- 
cal to Property 1, Property 2, Property 3, Theorem 3.1, Theorem 3.2, 
and Theorem 3.3 of Section 3.2 with the "r" of Section 3.2 replaced by 
"n" . 

In the following development, it is assumed that the true parameter 
0 q lies in the interior of a given compact subset (Eg of R^ n and that all 
systems with 0*<8lj are stable. The assumption of compactness is not 
truly limiting as parameter values, from a priori knowledge of the sys- 
tem, usually fall within a limi table range thus permitting the parameter 
set to be contained in a compact region. System stability is assumed 
to permit the investigation of asymptotic properties of the system. 

Under the conditions given, a characterization of the maximum like- 
lihood estimates follows. From Eqs. (153) and (154), 


1 L = 1 L + a" 1 (B l u l + E^). 


(157) 


Noting that T]^ * ^ - x^» the probability density function of the out- 
put ^ as parameterized by £ and z^ is 

rill,. - « L ha ♦ £ l vi' 2 

' 20 (1581 

A A 

Let 6. , 2 « t denote the maximum likelihood estimates of 0 and z_ from 
the observed data, u^ and ^ ; that is. 


P(Z L ii.£ 0 ) * Constant I exp [- ~|il L - ^(B^ + E^) {J 2 ] 


log p (i L ; -L’W » max log p(£ l ;0.,Zq) 


ZqC r 


(159) 
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For any £<8^, max n log is achieved by 

. *q«R 


*0L 




Thus, £ is obcained by 


min J. (6) - J T (fl ) 

Sc®. L L L 

where Che likelihood function J (_9) is 

J L d> - h L - A’ 1 [b l u l + I L io L (i)] II 2 
l|A. L £ L ■ — l— l * -L^OL^ II / v 

and 


( 160 ) 


(161) 


(162) 


^OL “ ^OL^l)* ^ 163 * 

It has been shown chat Che consistency of Che maximum likelihood 
estimates holds if Che almost sure (a.s.) convergence of the likelihood 
function is established. This property leads to the following Proposi- 
tions. 


Proposition 1 . 

For all 06(8^, J^(.9)/L — J(0,) with probability one, where 

J(6) - 11. i E J L (i) - O 2 + li» i ||» A - B L uJ|f A a T yl 
L L-*® 1 — L~L J 


and E denotes the expectation operator. 

2 

Note that J(0 q) * O' * min J(.6) which satisfies Eq. (153), namely, 
the true parameter vector, if 6^, is unique in the representation of Eq. 
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(153). In the following, the subscript "0" denotes the particular ele- 

ment at the true parameter value 9 q-, i.e., A^ Q » A^t^) , 5. L q * B^(9q' , 

etc. As seen in the next proposition, only those £ which give 

2 

rise to J(£) ■ J(0 q ) « 0 are of interest. 

Proposition 2 . 

A 

With probability one, converges to 

<B^ * [6. : J(0.) » J(6q)]. 

Proposition 3 . 

J(8) * J(0q) and only if 

U» i IK**, 0 - A - 0 . 

Proposition 2a . 

(An immediate consequence of Proposition 2.) Given that 6^ is a 

A 

unique, the maximum likelihood estimate converges to 6 q with proba- 
bility one if and only if is a singleton. 

A necessary and sufficient condition to insure that the condition 
of Proposition 2a is always satisfied is contained in the following 
theorem. 


— where 


Theorem 4.1 [11 

Given the linear dynamic system in any of its equivalent repre- 
sentations above, such that b 4 0 and (A,b) is completely controllable, 
the maximum likelihood estimate £. converges to 6.. with probability 
one if and only if 


lim 

L-. oo 


1 T 

T U T 0 U T O 

L — L, 2n— L, 2n 


> 0 
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where 


i L,2n 


[*iV 




The necessary and sufficient condition of Theorem 4.1 can also be 
stated in various forms for the purpose of different applications. 

Corollary 4.1 [11 

Given the linear dynamic system in any of its equivalent represen- 
tations above, such that b 4 0 and (A,b) is completely controllable, the 

A 

maximum likelihood estimate ^converges to 0 ^ with probability one if 
and only if 

lim 7 hV > 0 
. _ L -L-L 


or, equivalently. 


lim 

L-w 



where 




Theorem 4*. 1 can be viewed as the stochastic version of Theorem 3.2, 

and Corollary 4.1 can be viewed as the stochastic version of Property 1 

of Section 3.2 modified such that z^ 4 0; i.e., 0, "Hi* (x^ - E^z^ 

T 

if and only if H L is positive definite. 

this point, a link may be established between the deterministic 
parameter identif iability properties of the system and the stochastic 
parameter identif iabi lity properties of the system in terms of y ^ , the 
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noise-corrupted output, and M^, the Fisher information matrix. 

Theorem 3.2a f 201 

If the system described by Eqs. (148), (149) and (150) is stable 

T 

and z_ » 0, then 9 « [a. , . . . , a , b , . . . , b ] is identifiable, I.D.S. 
■- i n i n 

(equivalently, zero-state deterministically parameter identifiable per 

Section 3.1), if and only if the Fisher information matrix, M. , is 

Li 

positive definite for some finite L, where 

m l * S p(x L ;I»io) 1*0 L°g 

[*0 log P(X L ;i.zo)] Td V 

and 

X 

V _ „ / is_ 8s__ ds_ ds_\ 

9 Ida.’ ***’ da ’ 3b ’ ***’ 3b / 

“ \ 1 n 1 n/ 

for any scalar s. 

T / T\ -1 

By direct calculation, it may be shown that M L « I^AjJ H^. 

It may also be shown (Appendix B, [1]) that ^A^A^ is bounded such that 

Pi h * -L-L * p 2 h 


where 


0 < < P 2 <“ ♦ 

Thus, H?H t is positive definite if and only if R, is positive definite 
and, by Property 1 of Section 3.2, the system is identifiable, I.D.S. , 
if and only if M L > 0 for some finite L. 
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Note Che marked similar icy between Corollary 4.1 and Theorem 3.2a 
wich Che distinct difference Chat muse be positive definite for some 
finite L in the determinls-tic case while the limit of — must remain 
positive definite in the stochastic case. 

The Initial state does not effect the convergence properties of 
the parameter estimates However, the initial state estimate, as a 

A 

function of and is given uniquely by Eqs. (160) and (163); i.e.. 


^OL 


(i il' 1 C h)' 1 £ £' l (h ■ K 1 h 0 • (l64) 


Theorem 4.2 f 11 

If the conditions of Theorem 4.1 are satisfied, then E z^ "* Lq 
as L — ®, where E denotes the expectation operator. 


The system under consideration may be extended Co include an 
additive plant noise process which may be represented by the following 
Gauss-Markov model 


z_(j + 1) * A z_(j) + b_ Uj + d § j (165) 

x. « h T £(j) + b Q Uj + (166) 


where { 9 .} is a Gaussian white noise process identically, normally 

2 X 

distributed with zero-mean and variance X ; d ■ [d,, d-, ..., d ] ; 

— 1 £ n 

A, h, and b^ are defined as before. The unknown system parameters are 

a i* b i’ d i for 1 s i s n * b o and ^ 2; i* 6 ** 1 * [*> b > b Q’ ^ 

output is observed with additive noise 




<W7) 
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as before vich { T]^} independent and identically distributed, without 
loss of generality, as zero-mean, normal with unit variance. The noise 
processes { and ( T] are independent. 

Equivalent representations of the expanded system may be made as 
with the initial system but will not be presented at this time (see Ref. 
(1]). The following results were obtained. 


Theorem 4.3 f 11 

Given the system described by Eqs. (165), (166), and (167) with 
d 4 0 and (A, d) completely controllable, 6^ £ where 0_ ■ (a,b ,d,bg,X^)^J 
converges with probability one to 6. if and only if 


V 


lim 

L-» 06 


1 ~T 

- U U 
L — L,n — L,n 


> 0 


where 


2 L ,n ■ [v — L-L — L— l]‘ 

Theorem 4.3a [ 11 

Given the system described by Eqs. (165), (166) and (167) such that 
[A,(b,d)] is completely controllable, 9^ converges with probability one 
if 


where 


lim 

L-« 


lC2A,2,> 


> 0 




• • • , 




Selected proofs are presented in Appendix B. 
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4.3 Stochastic Parameter Identf liability Generalized from 
Constrained Maximum Likelihood (CML) Estimate Properties 

Using ths convergence properties of the meximum likelihood esti- 
aete, Tse end Anton [22] developed stochestic perimeter identiflebillty 
criteria expressed in terms of conditional probability densities for 
the sequence of system observation statistics. Their definition of 
stochastic parameter identlf lability remained the existence of con* 
slstent estimates as in Definitions 4.2 and 4.3. 

As before, denotes a sequence of observation statistics 

with a joint probability density function 2^; £), k ■ 1, 2, 

. .., parameterized by the unknown parameter c R p . Although the 

development by Tse and Anton was set in a more general separable metric 
space, in consonance with previous remarks, Q is taken as a compact 
subset of R p . The true parameter, is assumed to lie in the interior, 
of Q. An arbitrary norm on R P is denoted by ||*||. Denoting the obser- 
vation sequence 

l k m tXi* *2' •**’ l 168 1 

the sequence of true joint probability density functions may be denoted 
as 


P k (X l . •••> ^j) * P^Of^; 0^) , k ■ 1 , 2, ... ( 169 ) 

Since the context clearly indicates which density is indicated, the "k" 
subscript is routinely deleted, yielding p(Y k ; £q) • By Bayes rule, a 
conditional probability density function may be defined as 

p (*kl Y k-l* £) “ p ( Y k’ -) /p ( Y k-l* -) ' k - 1 , 2, ... 

( 170 ) 
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Any new Information obtained from the kth sampling will be contained 
in the conditional probability denaity of Eq. (170). For and p > 0 
define a regional conditional probability denaity 

Kviw *) • | fe . *?! t e < m > 

The following assumptions are made. 

Assumption 1 . The probability density function p(Y k ; £) is measurable 
in Y k with respect to p(Y 
almost everywhere; i.e., for any e > 0 and £* (1, there exists a 6(c) > 0 
such that for all cQ with - jl'\\ < 6 we have |p(Y fc ; §) 

- p(Y k ; $/)| < « for Y k almost everywhere. 

Assumption 2 . 

J' k log p (^, p|Y k-l ; i) p(Y k ; < • (172) 

R 

for each for some p > 0 and for ail k * 1, 2, and 

.f R k V V d \ < * <" 3 > 

for all k • 1, 2, . . . , . 

Assumption 3 . 

Var j log PCXjj.PlVr * 0<k2) (174) 

2 

for all £«£) and some p^ > 0 where 0 * P s Pq where 0(k ) is defined 

lira - 0 . 

k-«s k z 


k ; ^j)dY k and is continuous in £«() for Y 


such that 
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Assumption 4. Defining Che tec ft^S) • [v><v 1 ) - «], 

VS) ■ V£ ' 1 <l75) 

for £ ' <Q end for all k • 1, 2, . . 

ConcepCuelly, che second end chird estumpeiona rescricc Che growch 
race of eccumuleced in forme don ebouc che unknown paramecer relecive Co 
Che eccumuleced uncertainty. The fourch at sumption implies chec, for 
two different parameters, the corresponding density functions must have 
all Che impulses located sc Che seme points in Che observation space. 

Since Che only information abcuc is contained in che observation 
statistics with chair corresponding joint density function 

p(Y^; , k • 1, 2, if there exisc two parameter vectors &2 C ^* 

^1 ^ ^2 * uch 61146 

P(\i i x ) » P<\; io> (176) 

or 

p W Y k-l’ &l) * p (*lJ Y k-l’ - 2 )’ 411 k " l » 2 * ***• (177) 

Che Cwo parameters are indistinguishable In 0. 

Definition 4.4 T221 . Two paramecer s i.]_ ^ I 2 41 c to ** 

unre solvable if che equality 

K*kl*k-l! &1> • "<*kl Y k.l ; k> (178) 

holds with probability one for all except a finite number of 
integers k > 0; i.e., if Eq. (178) holds with respect to the 


1 


80 

measure p(Y R ; jj^)^ at well at p(Y k ; £ 2 )dY k< 

Definition A. St 221 . The tet Cl la said to be identifiable if wo two 
elements of Q are unresolvable. 

Definition 4.6 F 221 . For the observation sequence, Y k , the constrained 

A 

maximum likelihood (CML) estimate of ^ is defined as which 
satisfies 

P<V» L) • ®»* P« V J £)• (179) 

£cG * 

Essentially, this is the m a x i m um likelihood estimate of but 
takes into account the a priori knowledge that is constrained to be 
a member of 0; i.e., ^eQ. Since 0 is compact and p(Y k ; $) l'* con* 
tinuous almost surely by Assumption 1, at least one solution to Eq. (179) 

* m 

exists almost surely. Thus, the CML generated estimate sequence 
is a consistent estimate for ^ if ^ is unique, a consequence of the 
properties of the maximum likelihood estimate. However, if there exist 
two parameter vectors ** & 2 such tb4t 

11* P<\; 1,) - 11» P(\i i 2 ) (180) 

k-»* k • * 

A 0D * 

then, obviously, [^ k ] k-1 will fail to converge. 

Definition 4.7 [221 . Two parameters, j^eG, £ 4 are said to 
be CML unresolvable if 

11* ?%.[ * w i i r ) « 11* PdJVii < 18l > 

k ^ ® k ** ® 


with probability one. 
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Definition 4.8 [221 . The set 0 is said to be CML identifiable if no 
two elements in fl are CML unresolvable. 

Theorem 4.4 [221 . 

A sufficient condition for fl to be CML identifiable is that, for 
all 9^, 8 . 2 ®^* ^ ^2’ there an infinite set LCI + , l + the set 

of positive integers, such that the inequality 

V * KZkl’W k> 

holds with nonzero probability with respect to 0 ^ and 0 ^ uniformly in 
keL. 


It should be noted that Theorem 4.4 provides sufficient conditions 
to insure CML identifiability. If the conditions of the Theorem are 
not met, it does not necessarily imply that the CML estimation method 
will fail; rather, it implies there exists no guarantee of consistency 
of the estimate. In point of fact, certain studies have indicated 
that the CML estimate is "fairly" consistent even though some of the 
required assumptions are violated. 

Examp le 4 . 2 [231 . Consider the linear, time- invariant system given by 

5 J+l “ - s j + S J (182) 

1 ; « C x. + ^ (183) 

where x.eR n , y.eR m and | w. ) and I v.) are zero-mean Gaussian noise 
-J 1 "J 1 1 “J 1 

processes with covariances 
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and 

E [vj] ‘ 2 6 kj • 

T 

Let £ * [xq, A, £, R, 2., D] and assume that A is stable, (A,£) is an 
observable pair and (A,B) is a controllable pair, where 8 is the steady- 
state Kalman filter gain given by Eq. (188) (see below). 

Two parameters £^, ^ 0.2 are defined to CML unresolv- 

able if the equality 


p( \l' 


k-1’ -1 


ii) 


P(i k l Y k _l.i 2 ) 


(184) 


holds with probability 1 with respect to 9^ and 9^ as Since the 

0) is Gaussian 

a X 

with mean ^ and covariance £ £ £ + £ as k-*» (steady state). £ 

and ^ are given by the usual steady-state Kalman filter equations- 


system is linear and the noises are Gaussian, p (i^. | 


*k|k-l = - 2k|k-l 

\\k-l “ - ^k-l|k-2 + - ^k-l 

B = (A P £ T + D) (CP C T + £) “ 1 
P-APA T +R-B(£P£ T + £)B T 


(185) 

(186) 

(187) 

(188) 


( 189 ) 
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It can then be shown thac 9^, ^ ^ 9 2 are unreso ^ va * > ^ e 

if and only if there exists a nonsingular matrix T such that 


±1 

»ta 2 t' 1 

(190) 

h 

» T B 2 

(191) 


-1 


h 

* C 2 T 

(192) 


?! a] + 2 ! - c 2 ? 2 a] + a 2 . 

(193) 


If Eq. (193) is satisfied, it is clear that Eq. (184) holds with 
probability 1 as k-»«, and thus, 9^ and £ 2 are CML unresolvable. Then 
in the steady state, 

^l,k|k-l “ ^2,k|k-l 
with probability 1, for all k 

!=! ?! a] + 2, - c 2 p 2 4 + 2 2 - 

Equations (185) through (188) and (191) imply that 

C L aJ B - C 2 ^ B 2 , i * 0, 1, 2, ... (196) 

But Eq. (196) implies that the two steady-state Kalman filters [Eqs. 
(185) through (188)] have the same impulse response. Since (Au, C^) 
is an observable pair and (A^, B^) is a controllable pair, Eq. (193) 
results. 

Additionally, Glover and Willems [9] provide an example of a 
deterministic system driven by a white, Gaussian noise input. 


(194) 


(195) 
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Stochastic Parameter Identifiability from the Information Matrix 


Under essentially equivalent assumptions on the joint probability 
density function of the system observation sequence as found in Section 
4.3, Tse [21] has developed conditions for local stochastic parameter 
identifiability in terms of the information matrix. The system setting 
and terminology are also parallel to that of Section 4.3 


Definition 4.9 f211 . A parameter 8^eQcR H is said to be locally iden - 
tifiable if 

(1) There exists an open set Sq such that 0 q is an interior point 
of S Q ; and 

(2) There exists a consistent, local estimation sequence, 


| 0^ (Y k ; S Q ) J k=1 in S Q where S Q is the closure of S Q . 

The set Sq is said to be the region of parameter iden- 
tifiability. 

Considering the concept of resolvability of Definition 4.4, the 
Definition 4.9 is equivalent to stating that 0 q is locally identifi- 
able if there exists a neighborhood about 0 q, denoted by Sq, such that 
6 q is resolvable from its neighboring elements £«Sq. 


Theorem 4.5 f 211 . 

2 

If for all k » 1, 2, ... there exists \ > 0 such that 


»s 0 


I P 108 KiJWVi r * 108 
U as 0 JL J 1 


* x2 V 
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where represents the expectation operator with respect to the prob- 
ability density function p(Y^; -§q)» then Jq is l° ca Hy identifiable. 

The function ^(£q) is a conditional Information matrix. 

Vaguely, Theorem 4.5 implies that if there exists "positive information" 
about the unknown parameter in each new observation, then that para- 
meter may be recovered asymptotically provided that the region of un- 
certainty for the unknown parameter is small. 

Unfortunately, the condition of Theorem 4.5 is rather difficult 
to verify as it involves checking the positive definiteness properties 

of a countably infinite number of matrices. Further, it must be demon- 

2 2 

strated that these matrices are uniformly bounded below by X 1, X. > 0. 
Thus, it is desirable to establish a weaker sufficient condition by 
considering an additive,- and eventually total, information matrix 


J n (£> 

m,n 


'6 


I 

a log p (8) 

m.n 

d 108 ^m.n ( ~ > 

T 

1 

ae 

- « 

ae 

. 


(197) 


where 


Noting that 


I • * * ) 


*>• 


(198) 


and 


'9 


d log p (9) 

ai 2 


3 l °8 P m _<£) 


m^n 


ae' 


a log p m . n (D l P i<* p.,„<i) 1 T 
ae J L &£ J 

( 199 ) 


n a 2 log p (9) ' 

H™ — 
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we have 


n 

* E 9 

i*m — 


a log p . . (8) 



ae 


z J 


i*m 




( 200 ) 


J „<D 
ra,n 


.Z J. .(0) 
i*m i , i — 


(201) 


and 


J l,k^0* 


k 

.Z. J . . (JLJ 

i*l i,i -n) 


( 202 ) 


If the condition on ^(Sq) of Theorem 4.5 is satisfied then 

E J i, k <V * J *<V 2 x %x P ; ^ » 0 (203) 

w^ere J (0Lj) is also known as the average Fisher's information matrix . 
While Eq. (203) does not necessarily imply that the condition on ^(0^) 
of Theorem 4.5 holds, this weaker condition is sufficient tor local 
iden ti f iabi li ty . 


Theorem 4.6 [ 211 

P 2 

Let h be any unit vector in R . If there exists X > 0 such that 

lim £ h T J (©^h 2 X 2 (h) > 0, 

k- « ’ 

then 0 q is resolvable from 0, * 0 q + *h; 0 < c £ X(h)/c(h) for some 
constant c(h) < ®. Note that both X and c may be dependent upon h. 

Definition 4,10 [211 . A subspace QCR 15 is said to be locally identi- 


fiable if all elements 0efi are locally identifiable. 
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Theorem 4.7 [211 

A sufficient condition for a subset flCR p to be identifiable is 

that 

J*(8) 2 X 2 (0)I ; X 2 (0) > 0 for all 0eQ. 

While the results presented above appear to be particularly appli- 
cable to the analysis of a given system rather than to a class of 
systems, Tse [21] has applied the results to a linear, discrete- time, 
autonomous system presented in the following example. 


Example 4.3 [211 . Consider the system 


^+1 ” ^k *k (204> 

*k ‘ % % + \ < 205 > 


is a sequence of zero-mean, 
2 

independent, Gaussian random vectors with covariance a^I. The only 
unknown parameter is the initial state, £ « x^eR p . It can be shown 
that all required assumptions are satisfied [21] if the system is 
stable; i.e., \^\\ < ^ < ®, k - 1, 2, .... and if < c 2 <« . 

The conditional information matrix is given by 


where 


C, are known matrices and 

■TC 




where 




T T 


(206) 



*k ‘ k-V 


A.. 


( 207 ) 


Therefore, the total information matrix is given by 
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_ 13 -2 T _T 

J l,n — “ 4 kSl a k *k,0 k \,0 


for all 9*R P . 


(208) 


Suppose that the system is uniformly observable, 
k+n -1 _ 

J&c $j ik Cj E Jik * s 1. 9 > Oi J . 1, 2 


(209) 


and [o, ], , is bounded. Define 
k J k*l 


Vi • (v 


, •••' “(j+Un)' 


( 210 ) 


then, from Eqs. (208) through (210), 


J, 4 (9) * 7 0. n n for all 0CR p . 

l,jn — 4 i*l in,0 ^injO — 


( 211 ) 


Therefore, from Theorem 4.7, a sufficient condition for x^ to be iden- 
tifiable is 


lim t 

i -» 00 ^ 


J 

iSi 


*-2 T ^ ,2 t .2 

a i ^n.O ^n,0 2 X X > 


0 . 


( 212 ) 


Note that, since the system is linear, the least square estimate 

has error equal to J 1 (£) exactly. Therefore, a less restrictive 

l,n 

sufficient condition will require 


°i " 2 *in ,0 Sin.O - *• (213) 

In an attempt to more directly relate the above results to systems 
notation, consider three systems in which the parameters £ are linear, 
nonlinear, and dynamic-nonlinear functions of the observations 2 
the presence of independent, Gaussian noise v with covariance matrix 
Q. The three representations are, respectively: 
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j ■ H 0 + v 


£ - H Z (£) + V 


(Linear) 


(Nonlinear) 


y(t ) • H z (t, ,0) + v(t.) . . 

^ i — — i — — i (Dynamic -Non linear) 


(214) 

(215) 

(216) 


The form of the information matrix may then be determined for each 
case. Consider first the linear case of Eq. (214) which implies that 
X " H d, is distributed as v. Then: 


log p(x|l) = Constant + - H £] T £”*[X - H 0J 

» y -ll-uV 1 ! 

jp log p(i|e)jp log p(xi£)j^ j 

- E j H.V” 1 ti - H 0Ui - H 9] V 1 H 


(217) 

(218) 


T -1 
H a H. 


For the nonlinear case: 


(219) 


log p(x|£) * Constant + - H z,(£)] T 2 ^X ’ £ £(£)] (220) 


d log p(yjd) 


, T -1 


8 z (6) 


ae Ix-Hz^i^^H-gp 

fra log PdlDira log p(x|6)-| ) 

(l ae JL 51 ”J^) 
p *(0) ] T T , ra i(i>l 

“L“5i J * a 


(221) 


( 222 ) 


For the dynamic-nonlinear case: 
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log P(xll) * Constant + j | " H z(t t ,9)] 

ja " 1 Ci.C t £ ) “Hz (t if i)] J 


(223) 


d log p(i|9) K T l S 5 .Ct ,9) 

“ iSi - H 2(^,9)]^' EL ^ 


36 


(224) 


{[ 


3 log p(i|9)ip3 log p(x|9) 

Si JL 36 


]li) 


k fa z(t ,9)1 fa *(t ,er 

ili [ al J & a h L — gg— _ 


(225) 


It should be noted that these exact expressions appear in small 
variational parameter estimation methods as the gradients of the quad- 
ratic cost functionals. Likewise, a minor link may also be established 
to deterministic parameter identifiability by noting that the cost 
functional for least square identifiability may be taken in the sto- 
chastic case to be the negative logarithm of the likelihood equation. 
Then, the expected value of the second partial derivative of the cost 
functional with respect to the parameters is the information matrix 
discussed above. 


4.5. Comments on Stochastic Parameter Identifiability 

While the injectivity of the function £ provides a unifying set of 
concepts and definitions for deterministic parameter identifiability, 
no corresponding unifying concept or definition has yet been deter- 
mined for stochastic parameter identifiability. A loosely unifying 
concept for stochastic parameter identifiability appears to be the 
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existence of a consistent estimate for the unknown parameters. However, 
the requirements for the existence of a consistent estimate is express* 
ed mathematically in a number of different ways and thus no mathemati* 
cally uniform concept now exists which is equivalent to the functional 
Injectivity requirement for deterministic parameter identif lability. 

As shown in the previous material, several tentative links between 
the forms found in deterministic and stochastic paramer'r identifiable 
ity have been established. However, no mathematically explicit, con* 
sistent relationships have been established to relate the two concepts 
together. 

It should be noted, however, that deterministic parameter identi- 
fiability is a prerequisite for stochastic parameter identlfiability. 
Indeed, if a system is not deterministically identifiable, then certain- 
ly no consistent estimate for the unknown parameters can oxist. On 
the other hand, the assurance of deterministic parameter identifiabili- 
ty is not sufficient to insure stochastic parameter identlfiability 
since the stocnastic properties of any given system may supercede the 
deterministic properties. 

It should be noted that the final two stochastic parameter identi- 
fiability concepts presented in Section 4, specifically, those predi- 
cated upon the properties of the conditional probability density func- 
tions of the system observations and those predicated upon the proper- 
ties of the conditional and total information matrices, are oriented 
towards the analysis of a given specific system rather than of a total 
class of systems. The application of these concepts to some classes of 
systems of Interest seems to be an opportune field for study. 
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APPENDIX A. PARAMETER 



Characteristic 

General, Noe 

System Specific 

Oe termini* tic 

Stochastic 

Linear 

Nonlinear 

Continuous 

Qiscrete 

SiSO 

MIUO 

x Known 

•■o 

x Unknown 

“O 

Unspecified Input 
Zero Input 
Impulsive Input 
Local 
Global 



8 8 8 8 

s s s s 

* « « « 

£ f f £ 


X X X X X 

X X X X X 

X 

X X X X 

X X X X 

X 

XXX X 

X 

XXX 

X 

X 

X 

X X X X 


Theorem 3.4 
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X X X X X 
XXX XXXXXX 

XXX 

xxxxxx 

XXXXXX 

X 

XXXXXXXXXXX 
xxxxxxxx XX 

x 

xxxxxxxxxx 

X 

xxxxxx 

X X X X X 


C-iL 


Iheori 
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APPENDIX B. SELECTED PROOFS 


The proofs selected for presentation in this Appendix are not 
totally comprehensive but are presented as samples of the methods and 
procedures involved. Generally, proofs which require the establish- 
ment of preliminary lemmas or proofs have been omitted. Other proofs 
which are similar in content and nature to those chosen have also been 
omitted. 


Theorem 3.1 [201 

From the defining Eqs. (35) through (37), direct computation 
yields 


lA 


* x n+l* X L-l] “ [v Vi’ **•’ a l] 


n-1 

T 


T 

h A 


x. . . . x. . 
1 L-n-1 


X. ... X— 
l L-n 


X . . . X- - 

n L-2 


[v - “O 1 


h T A< B - l) 


t a , a . 
n’ n-] 


. *1 


' • -r 


]• 


(B.l) 


The first matrix on the right is non-singular by Eq. (36). Therefore, 
[a^, •••> a ^J i8 uniquely determined if and only if the second 


matrix, the controllability matrix for the pair (A,z ), has rank n and 
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the theorem follows immediately. 

Theorem 3.2 r 201 
Necessity. 

(1) H L is L X (n + r). h£h l > 0 implies L * n f r. 

(ii) If s 0, then * 0, x L « A' 1 ]^ - 0 

where Eq. (37) is restated as 

-L-L “ -L-L + -L^o (B*2) 

and 


±L ■ h * A a j4 < b - 3 > 

and 

ll ■ • < B ' 4 > 

Consequently, any satisfies Eq. (B.2). 

(iii) If u. 5 0 for O^iSL-n-r, then by Eq. (35), x i s 0, 

also, for O^iSL-n-r. Thus, the first L - n - r + 1 rows of 

T 

become zero and rank < n + r, and *■* not P 08 ^!^ definite, 

(iv) If A(z) and B(z) have a common divisor, then 


— L 


Ml* k’hk 


(B.5) 


— L 


k 


n 



j£l *j-L’ n l * n 


where 
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and 


—L 


n- n i 

dl, + ,1. d.S T J , d 4 Q. 
o-L j»l j— L’ o 


-1 


Substituting Eq. (B.5) into Eq. (B.2) and multiplying both sides by 0. , 


since 4 0, we obtain 


k. k * Vl ; 


then ? « , a 2 , . . . , a R 


» 4 ♦ * » i * • * t 0 J 

1 n l J 

would satisfy Eq. (B.2) and contradict the uniqueness assumption. 

Sufficiency. Let 8,- ^a^ a 2> ..., a^, S, , b 2 .... b^J 1 be any 
vector such that the corresponding matrices A. , B satisfy Eq. (B.2). 

“ L “L 

Then • \h. l , iA - ra “ ef!,r '- 


" — tAi.— L 


kkk 


MA 


Let 


“L 


— L-L 


-Ml 


n+r 

A 


c - S l 

J“L 


(B.6) 


(B.7) 


Let 


U T 

-L,n+r 


[*lV ^L' 



(B.8) 


and 


c -[c 


1 * 2 * 


n+r 


]'■ 


(B.9) 


102 


Combining Eqs. fB.6) through (B.9), we obtain 


U, c ■ C T u. * 0. 
~L,n+r— — L— L 


By (i) and (iii), U. has rank n + r, 
“L ) n+r 


Ml " Mr 


(B.10) 

Hence £ » 0 and, by Eq. (B.7), 


This implies B(z)/A(z) * B(z)/A(z) *nd, by (ii) and (iv), a * a and 
b « b. This completes the proof. 


Theorem 3.4 [9] is an immediate consequence of the definitions 
given and is an application of the constant rank theorem for injective 
maps [ 17 ] . 

Theorem 3.6 [10] results immediately from the application of the 
variation of constants formula 

Z(t,6) * C(0) [ e^ ( - )(t ' T) B(0)u(T)dT + D(0)u(t) (B.ll) 

J 0 

and Definition 3.8 of distinguishability. 


Corollary 3.6 [101 

* C(8 2 )A*(e 2 )B(6 2 ), L * 0, 1 , 2, ...(B.12) 
0(6^ ■ D(9 2 ) (B. 13) 

Sufficiency. Equations (B.12) and (B.13) imply that the two 
systems corresponding to the pair of parameter values both 

have identical impulse responses and hence have identical transfer 
functions. By Definition 3.8 the pair of parameter values is indis- 
tinguishable. 
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Necessity. By Definition 3.8, the India tinguishsbi li ty of 
(£^,£ 2 ) implies for • 0, for any given u(t)c(i and for all 0 £ t £ T, 
that 


^(t.0^ - l(t,0 2 ) 


which, by the variation of constants formula, becomes 


w 

£(8.) e- ( -l )(t ‘ T) B(0.)u(T)dT + £(0,)u(t) 

1 Jo 1 • 

■ C(£ 2 ) e- ( -2 )(t ' f) B(£ 2 )u(T)dT + D(0 2 ) u(t). (B.14) 

Since £q. (B.14) holds for all 0 £ t £ T, and in particular for- t * 0, 
then 

£(1^(0 " D(0 2 )o(t) 


for all u(t)cil and for all 0 £ t £ T. Clearly, 


0(0^ 8 D(0 2 ) 


(B.15) 


and 


0(0^ | e - ( ^l )(t ' T) B(0 1 )u(T)dT 

«£(0 2 ) J e- 2 


(T)dT. 


(B. 16) 


Combining like terms and moving C(6^) and C(£ 2 ) under the integral, 
since they are independent of T, yields 

f Q |&(£ l )«^ ( ^l )<t “ T> l(£ l ) - C(0 2 )e- ( -2 )(t ’ T) B(£ 2 )J u(T)dT a 0. 
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Since Che functions in Che equation above within the brackets and u(t) 
are continuous, by the standard theorem of the calculus of variations 

£<i 1 )a^ < ^l )<t * T) i(i i ) - C(6 2 )e-^2 )(t ‘ T> B(0 2 ) - 0. 

By repeated differentiation with respect to t and evaluation at t » T 

Sd^id^ * C(6 2 )B(£ 2 ) 

CCi^Ad^B^) = C(9 2 )A(0 2 )B(9 2 ) 

a a 

* a 

a a 

Cdx^diddi) ■ £d 2 )A £ d 2 )£d2)» * - °> 1. 2. ••• 

(B. 17) 


as required. 

Theorem 3.7 [10] follows immediately from Corollary 3.6 and Defi- 
nition 3.9 for (local) parameter identif lability. 

Theorem 3.8 [10] is an immediate consequence of the given defini- 
tions, Theorem 3.7 and the constant rank theorem for injective mappings 
[17]. • 

Theorem 3.9 f 101 

It has been shown [6] that the solution to Eqs. (97) and (98) can 
be written as 

te*.6 0 .6£ll • ItooKio’ 6 ^ + 

* lta 0 (‘d 0 .fie>|| ' lk (, ’ 6 £ ) il (B * l8) 

where ^(*,60) are terms of 0(||6(9)||) and ^.in^ 0(6)/6 » 0. 
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Defining th« L^t^.T) norm* of Bq. (B.18) yields 


U6x(T,S 0 »fiil| 2 < lT 


[ T iht(T.flfl>if 


1/2 


T 

j '(^(T.So.ailfdT 


1/2 


'dT 


1/2 


(B.19) 


Divide boch sides of Eq. (B.19) by ||66||. Assume Chet ||6g|| lies between 
0 < ||6£|| < 6. Then 


V / Ite^.So.Sill Vdr' 

1/2 

'(i /lte 0 < T .S 0 .«i>ll \ 2 

l t v imh — J 

- 0 

2 

\ ' M j dT 

. 0 


fi /lk(T,6e)|| \ 

\m\ ) 


2 11/2 
dT 


(B.20) 


Now assume 


j dT > 0, 


and 


min 


{j* [iV ^(T, V ] 


dT J ■ « >0 


where k , is the minimum eigenvalue, 
min 

The first term on the right-hand side of Eq. (B.20) can be written 


as 


(T 6£ 66 

) M U» (T .V!i< T -VMil dT 

0 


for 


1/2 

i c > 0, 
all 6. 


(B.21) 
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At 6-*0, Che second term on Che right-hand tide of Eq. (B.20) 
tends co zero, and Che cocal right-hand side is greacer chan zero for 
all 6 sufficiently small, say 6*6. Therefore, che left-hand side of 
Eq. (B.20) is 


[c.( 


— ra — 


) 


T 1/2 


dT 


> 0; 


(B.22) 


hence, ||6l(* ,^,68) || 2 > 0. Hence, for all 6 4 SeS^.fi), £(t,0) 
4 x(C,6q) for some tcic^.T], which implies chat che parameters, Cl, 
of the nonlinear system are locally identifiable at 6^. 


Wald demonstrated the consistency of the maximum likelihood esti- 
mate by first establishing three lemmas which are presented below with- 
out proof (see Ref. [24]). 


Lemma B.l. If £ 4 9^, then 

E log p(y;6) < E log p(y;6 (1 ) 


(B.23) 


Lemma B.2 . 


lim E log p(y;8,p) ■ E log p(y;6L) 

p-0 


(B.24) ( 


Lemma B , 3 . 


lim E log f (y ;r) • - * (B.25) 

r 

With the above lemmas and the law of large numbers, the following 
theorems, leading to the consistency of the maximum likelihood estimate, 
can be proven. 
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Theorem B.4 

Lee UCQ be a closed subset of Q. If ^ does not belong Co tf, 

Chen 


Pr 


•up p(y.,y, y. ;£) 

Qcw 1 2 “ 

lim — ■ . ' a \ ■ 0 i 

k— vV J 




Proof: By Lenna B.3, we can choose r Q > 0 such chac 


2 log ¥(y,* 0 } < 2 log p(y;£). 


Lee be Che subsec of U such chac 


W, Sew}. 


For each 0cW, , we can choose a p 0 > 0 such chac 
- 1 2 


E log p(y;£,p^) < E log p(y;i 0 ). 


(B.26) 


(B.27) 


(B.28) 


The exiscence of pg is guaranteed by Che law of large numbers and 

B.l. The sec W. is closed and bounded and, hence, is compacc. 
Thus, Chere exists a finite number of points 2}, •••> £j 1° such 

chat Che union of Che spheres wich center £. and radius p fi , i • 1, 

j 

.... j. t y t s covers w i* 

It is seen Chat 


0 * sup p(y. , y-, ..., y.;0) 
6cU 1 2 K 


iii p ( y i i ^i* p a t ) 


* f<y 1 ' r o ) ^ 2 k * r o ) * 
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It is then required to show that 


and 


Pr 


pOtiyoft) ••• Kyfit.Qtt) I , 

••• ?<vV " I " ’ 


11m 
k 


(B.29) 


1 • 1 , « . . » J * 


Pr 


lim 


llvv 


•1W 


k-. P<ySo> P<W 

which is equivalent to showing that 


(B.30) 


Pr 


i?.ii h p (w%) - i °» Kvy)] 


i, 


i * 1* •••* J ; (B.31) 


and 


Pr 


' k r 

{lim Z \ 

{ k-*« m«i L 


l °8 f(y B ;r 0 ) - log P(y m 


>v] 


» 1.(B.32) 


But Eqs. (B.31) and (B.32) follow immediately from Eqs. (B.27) and 
(B.28) and from the (strong) law of large numbers. 


Theorem B.5 


^ k (y 1 , * . . > 


y k ) be a function of the observations ouch that 


p(y,. •••. 

“ 7 “ v i c > 0 for all k and for all 

p(y^« ...» y k >2Q> 


'V 


• * • i y 


k* 


(B.33) 


Pr 


lim e, 

k -• oo 



1. 


Then 
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Proof: Lee ® denote the set of limit polnte of Then it 

suffices to shot* thsc for any « > 0, 


sup | ||£ - £q|J: * « with probability one. (B.34) 

Suppose chat there exists a £*® such chat - ^]j > s, then 

\k “Lii 1 * P<) ' 1 v4> * p<yi 


for infinitely many k. But this implies 


sup 

-Soil 


a c 


p(y^ * •••* y k »£) 


p(y lt 


■ ♦ • . 


y k ; V 


a c > 0 


(B.35) 


for infinitely many k by Eq. (B.33). By Theorem B.4, Eq. (B.33) is an 
event with probability zero; thus, Eq. (B.34) holds with probability 


one. 


Recall that the ma x im u m likelihood es 

p(y,. y v ;L) • ®* x p(y,. • • 

1 * i«u 1 


tlmace ^ is obtained by 
•i y k ;£). k • 1. 2, . ..(B.36) 


If exists, chen 


y . ^ i * y ^ » • • • • y k i k«i,2, ... 


p(y i W 

p(y lt • 

Clearly, by Theorem B.5 , the maximum likelihood estimate la con* 
sis tent. 


Proposition 1 (pace 71) . In Eq. (162), the vector ^ has only a 
finite number , n, of nonzero elements. As L-*», it contributes nothing 
to J^(£)/L in the limit. Therefore, can be dropped from Eq. (162) 
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without lota of generality. Than 


1 ■ I I Ml * Ml " Mi. if . 

(Ml) 

* lII Ml ‘ MlII 

MY 1 

*i £ [si - ^‘ia] ♦ Kii 2 - 


(B.37) 


(B.38) 


Iht firtt ttrm is deterministic. In tht second tarm, both 5^ 

raprtsant output taqutncct of tomt stable system with s bounded 
input sequence. Thus, x^ - A’^g^u^ is uniformly bounded. The limit in 
the Proposition statement |llm £ E J. (£)1 exists, end 

Ll— J 

L [-L ' “L *L-l] ‘ l A a i^i’ (B.39.! 

where * a < • for ell i. Since |l]^| ere independent random variables 


with 


and 


ETj t • 0, ET1* - 0 2 < * 


z 4 * a 2 s 4 < «, 

i t 1 i 


the sarong lew of large numbers applies. Hence, with probability one, 


7 S - 0 


1 

L 


- -2 2 
z tj, - 0 , 

1 • 


(B.40) 


(B.41) 


Ill 


and substituting Eqs. (B.39) through (B.41) Into Eq. (B.38), we have, 
with probability one, 


£ J L (0) - J<0). 

Theorem 4.1 f 11 

Sufficiency. Define the matrix C, and the associated vector £ by 


" -L-L.O " -L,(£l 
2n 

* S C i^* 
i-1 1 L . 


and 


= - [«, 


■ c 2* c 2n] 


Then, 


0. u» l|(4A >0 - 

L-* 09 

if and only if 


lim 

L-» «9 



T r IT 

1 1 -L.2A,2n£-J 


0. 


The limit condition of the Theorem statement; i.e. 


lim 

L-* 99 


1 

L 


U U 
-L,2n-L,2n 


> 0 , 


implies that £ ■ 0, or, equivalently, 

C(z) * A(z)Bq(z) - Aq(z)B(z) » 0, 
B(z)/A(z) ■ B 0 (z)/A Q (z) , 


(B.42) 


(B.43) 


(B.44) 


(B.45) 
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where 

A(z) ■ 1 + ajZ 1 , B(z) » ^ b i z l . 

Hence, by controlability, b « b^, a « a^, £ » 0^, and ©s^o is a 
singleton. 

Necessity. It suffices to show that if the limit condition of 
the Theorem statement, Eq. (B.45), is not satisfied, then there exists 
£ - (Sq + 6a, bg + 6b.) such that 6a 4 0, 6b ^ 0, the condition of 
Proposition 3 is satisfied, and £«®b'"'© s * 

Note that the vector £ as defined in Eqs. (B.42) and (B.43) can 
be re-expressed as 


c * T, a + T b+E 0 b, 
— “ho — — ao— — 2n— o 


(B.46) 


where 


T « -A, E 0 . (2n X n); 
— ao — 2n,0 — 2n’ 


ho m h«,ohn’ (2nX,l >> 

, (2n X n) . 

If the matrix V is not positive definite, 


— 2n 


I 

-n 


n,n 


Let V - lim f uj , U. 

T L — L,2n — L,2n 
L- 00 

then there exists a nontrivial solution to the equation 


0 - V (T^, Tj 


6a 

6b 


(B.47) 


It is immediate from the definition of C that C n * 0 and thus 


0 ■ T. a + T b +E«b. 
~bo~o -ao-o —2n~o 


(B.48) 


Therefore, letting a « a^ + a * 6a , b » ^ + a • 6b for any scalar a, 

X 

we obtain by combining Eqs. (B.46) through (B.48) that £ V£ • 0 which 
by Eq. (B.44) implies that the condition of Proposition 3 is satisfied 
and lb only remains to show that 9c(Q^. 

Suppose [X. i (A)] > 1 s i s n; are the roots of A(z). Then, ^(A^ 
are exterior points of the unit disc D » ( z: |z| £ 1 } on the complex 
plane by stability. Since the roots of A(z) are continuous in a at a^ 
in the sense that there exists a neighborhood^ where (0)^ * {a: 

| (a - a^H <c } such that \^(A)«D for all ac^, clearly a « a^ + a • 6a 
is stable if a < */||6aj|. Furthermore, 8^ is an exterior point of 
Thus, d can be chosen to have 


0 * (a + a • 6a, b + a ♦ 6b)s<8> . 
“ “O “ “O s 


(B.49) 


